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The Next-Generation IT Infrastructure 


Cyclades AlterPath™ System is the industry's most comprehensive Out-of-Band Infrastructure (OOBI) system. The AlterPath 
System allows remote data center administration, eliminating the need for most time-consuming, remedial site visits. When fully 
deployed in your data center, Cyclades AlterPath System lowers the risks associated with outages, improves productivity and 
operational efficiency, and cuts costs. 


Each component of the AlterPath System is designed to seamlessly integrate into the enterprise, able to scale in any direction. 
Whether you need serial console management of networking equipment, KVM for access to Windows® servers, branch 


management, IPMI or HP iLO for service processor management or advanced power management, the AlterPath System delivers. 
Cyclades brings it all together, making OOBI administration seem like child’s play. 


Over 85% of Fortune 100 = 
choose Cyclades. 
www.cyclades.com/Ija 
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8x AMD Opteron Processor 940 sockets 

Supports 800 series Opteron CPUs with dual core tech. 
Up to 128GB DDR Registered ECC memory 

Support 4 Ranks memory module 

1350W Redundant PSU 3+1 

Support IPMI server management 

Industry 19" rack-mountable 5U chassis 

4 x Gigabit Ethernet ports, and 4 PCI-X slots 

Up to 10 hot-swap HDDs with option HDD canister 
Modularization design, I/O may vary 


8-Way AMD Opteron Server Benchmark Rating 
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2 AMD Opteron Processor 940 sockets * 2 AMD Opteron Processor 940 sockets 2 AMD Opteron Processor 940 sockets * 2 AMD Opteron Processor 940 sockets 


* Supports 2xx Opteron CPUs with dual core tech. 

* Up to 16GB DDR Registered ECC memory 

© 1x PCI-X 64bit 133/100/66MHz expansion slot 

* 2x PCI 64bit 100/66MHz expansion slots 

¢ HTX-Pro support Pathscale InfiniPath HTX Adapter 
¢ 2x Gigabit Ethernet ports (Intel chipset) 

¢ AGP 8MB on board & support IPMI 


* Supports 2xx Opteron CPUs with dual core tech. 


* Up to 16GB DDR Registered ECC memory 
* 3x PCI-X 64bit 133/100/66MHz expansion slots 
* 2x PCI-Express x16 expansion slots 
(one in PCI-Express x2 mode ) 
* 2x Gigabit Ethernet ports (Broadcom chipset) 
¢ AGP 8MB on board & support IPMI 


Supports 2xx Opteron CPUs with dual core tech. 


Up to 64GB DDR Registered ECC memory 

2 x PCI-Express x8 expansion slots 

1 x PCI-X 64bit 133/100/66MHz expansion slot 
1 x PCI-X 64bit 100/66MHz expansion slots 

2 x Gigabit Ethernet ports (Broadcom chipset) 
AGP 8MB on board & support IPMI 


* Supports 2xx Opteron CPUs with dual core tech. 

* Up to 4GB DDR Registered ECC memory 

* 1x AGP 8X, 1x PCI and 1x mini PCI slot (ZMAX-DP) 
* 2x PCI-Express support SLI Tech.(ZMAX-D2) 

* 3x 3.5" HDD bays, and 1x 5.25" CD-ROM bay 

* 1x GbE, 1x IEEE1394, 8x USB 2.0 ports 

© 300W Power supply 
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66 PROJECT UTOPIA 
Linux’s long-standing tradition of isolating the user from the hardware might be 
great for security, but it can be a real pain when you just want to snag some 
photos from your camera or check for wireless access points you‘e allowed to 
use. But there is a plan. This month, Robert Love covers Project Utopia. 
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NetworkManager gets notifications of new 
network hardware and available access 
points, so all you have to do is find a hotspot 
(page 66). 
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HACK ANYTHING 


Internet radio doesn’t have to tie you 
to the computer. Dan Rasmussen, Jon 
Morgan and Paul D. Norton have 
updated a classic radio design with 
the ability to tune in Internet streams. 


Stuart Brorson covers the electronic 
design automation tools needed to 
work with schematics and crank out 
professional-looking board designs 
that you can build yourself or order 
from a PCB house. 


If you think your favorite pinbal 
machine is complicated now, try 
interfacing it to a Linux box. John Bork 
covers digital I/O techniques to help 
you connect to useful devices such as 
solenoids and switches. 
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rm -rf /opt/bs 


A Linux desktop shouldn’t be a kick in the teeth. 


BY DON MARTI 


o understand the IT indus- 
try, start with On Bullshit 
by Harry G. Frankfurt. Prof. 
Frankfurt poses, but doesn’t 
answer, the question of why there is 
so much B.S. in our society. He 
compares his subject to shoddy con- 
struction, and that’s an analogy we 
can work with, because in software 
we’re working at the thrilling edge 
of language and craftsmanship. We 
have the tools for dealing with B.S. 
in computer languages. Try to B.S. a 
compiler and that’s a bug. It’s time 
to tackle the B.S. problem head-on 
and start reporting bugs in human 
communications too. 

Consider this filler, I mean 
essay, to be a bug report on the 
big companies that are doing 
Linux for the desktop. “Let’s 
‘position’ Linux as a simplified 
desktop for ‘transactional users’”, 
they say. That’s right—employees, 
if your company gives you Linux, 
that means Management thinks 
you’re a human servlet. Decision- 
makers and content creators get a 
proprietary desktop OS. 

Of course, offending the 
employees’ pride might not show 
up on a TCO spreadsheet. But no 
executive would want to admit to 
running a division full of transac- 
tional, replaceable, outsourceable 
“human resources”. 

But what about Clayton 
Christensen, disruptive innovation 
and The Innovator’s Dilemma? 
Doesn’t the cheap, good-enough 
contender always grow the fea- 
tures and stability it needs to win? 
Yes, when it lets in the customers 
left pressing their noses against 
the Expensive Stuff Store window. 
In the 1980s Macintosh let you do 
layouts even if you couldn’t afford 
phototypesetting. In the 1990s 


Linux let you put up a Web server 
without blowing the price of a 
Coupe de Ville on a UNIX box. 

But selling less-capable prod- 
ucts to customers who can get the 
good stuff doesn’t fly. Seen an 
F-20 at an air show lately? It was a 
capable airplane, but it was posi- 
tioned as an “export fighter” for air 
forces that weren’t allowed to 
have, or couldn’t afford, the F-16. 
Naturally, countries held out for 
the “real” fighter. Information free- 
dom ideals can go only so far when 
vendors patronize Linux cus- 
tomers. “Aww, the little transaction 
worker filled out a Web form! Isn’t 
that cute?” 

Desktop Linux marketing is 
doing more harm than good, but 
work is under way to make Linux 
out-perform the other OSes. Robert 
Love’s Project Utopia is bringing 
together the desktop interface and 
the necessary tweaking of hardware 
to make things work smoothly, not 
just securely (page 66). 

Michael George has an example 
of how a thin-client environment 
almost works to solve a problem, 
but the project needed one key 
local app, the soft phone. See a 
hybrid approach to a VoIP station 
that works as a phone and a PC on 
page 72. 

One of the projects where soft- 
ware excellence, not transaction- 
workerism, has triumphed, is 
Mozilla Firefox. Mozilla expert and 
author Nigel McFarlane died last 
month, leaving us with one last 
article (page 52). Let Firefox serve 
as an example for the standards the 
desktop is coming to meet because 
all B.S. aside, it has tol 
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The Power of Choice... 


Modularity. Scalability. Reliability. Cost-effectiveness. S 


These represent the solid foundations that SBE delivers to 
OEMs for building innovative end solutions. Partnering with 
SBE for networking and communications 1/0 solutions allows 
you to take advantage of proven technology and field-tested 
products designed to optimize performance for your unique 
application needs. 


SBE offers a full spectrum of interface cards, ranging from T1 
and T3 to Gigabit Ethernet and IPsec/SSL acceleration. These 
boards are available in multiple form factors, including PCI, PMC, : 

and PTMC. Customers have the choice of buying these boards > Channelized T3 
individually or bundling any of the PMC/PTMC modules with our > 24-port T1/E1 > IPsec/SSL Encryption 


intelligent core processing platforms to create a flexible, cost- 
efficient blade solution ideal for serving demanding telecom > LAN/Ethernet > Blade platforms 


applications. Full Linux support is available on every board. > Storage > I/O and beyond... 


7 Linux On Demand 


flexibility on demand | 925-355-2000 | info@sbei.com | www.sbei.com 


LETTERS 


Ultimate Linux Box Cooling? 

— ET 
I was impressed by the desire to make a 
quiet PC—more people should complain 
to their OEMs/System integrators about 
this—it is the only way it will be fixed 
[“Ultimate Linux Box”, August 2005]. But 
I am not sure that removing 100% of the 
airflow is a positive thing for overall sys- 
tem performance or stability. 


I have seen motherboards designed where the 
processor power supply components can 
exceed the design rating from the suppliers 
without airflow. Even if things don’t go bad 
enough to cause system stability issues, it can 
damage the processor by allowing the CPU 
voltage to go out of specification. Intel is so 
concerned about this—they are telling moth- 
erboard manufacturers to add circuits to the 
motherboard to monitor the temperature of 
the processor power supply and modulate the 
clock of the processor if things get too hot. 
See Section 9.4 in download.intel.com/ 
design/Pentium4/guides/30235604.pdf. 


Thanks—and looking forward to more sub- 
minute kernel compiles. 


Robin 


Whenever you experiment with any alternate 
cooling method, always measure and log 
temperatures.—Ed. 


Linux/BSD Confusion 
ee ees 
My child is almost two in the picture. The 
laptop is running KDE on FreeBSD. He 
quite plainly CALLED it “Linux”. My wife 
and I both looked at each other and at him, 
and he said it again. He has also said 
“Ethernet”. We are afraid, very afraid. 
Thanks for a (decade of a) great publication! 


kurtseel 


More on the ULB 

a ee] 
I’m used to reading the annual Ultimate PC 
article in Maximum PC magazine each year, 
and they give a great deal of detail and 
many more pictures than the Ultimate 
Linux Box article [August 2005]. However, 
the big difference between the two articles 
is that your Ultimate Linux Box does much 
more customization. 


The big question is, if I want to replicate the 
Ultimate Linux Box, where would I get 
details on the custom-made power supply 
cooler modifications? 


I went to the Resources page, but many of 
the things I would need to access require that 
I’m a Linux Journal subscriber. I have sub- 
scribed to the Linux Journal in the past, but 
for the last year, I simply run out and pur- 
chase Linux Journal. 


Dean 


Articles from that issue will be openly avail- 
able soon. We'll look for more info on the 
power supply mods.—Ed. 


Ten Years of Progress 
SSeS 
The shock has worn off from seeing my 
name in Linux Journal, and I’m able to 
write again. [See “Ten Years Ago in Linux 
Journal’, July 2005, page 14.] Has it been 
ten years since “Novice to Novice” 
appeared in Linux Journal? It must be. I 
stopped writing the column after my first 
child was born and—bless it!—if it’s not 
his tenth birthday already. 


And how much has Linux changed (and 
stayed the same) in ten years. Though I 
haven’t quite made the switch away from 
Microsoft, I did recently install Fedora 
Core. In ten years, installation has vastly 
improved since when I used version 0.99 of 
Slackware. The Fedora installation was rela- 
tively fast and idiot-proof. Everything 
worked except the modem, and although 
I’m finally getting DSL installed, I wanted 
modem access as backup. Turns out I have a 
PCTel modem, which seems unsupported by 
the 2.6+ kernel and by the drivers currently 
out there in Webland. 


(Hmmm, could be another “Novice to 
Novice” here.) 


But what’s blown me away about Linux are 


the live CDs. Knoppix and the variations are 
fantastic not only for emergencies but also 
just to learn *nix, which is why I got 
involved with Linux originally. 


Ah well! Thanks for remembering me after 
all these years. Yes, I still have the 0.99 
Slackware CD with the grinning Bob. It just 
seems right to keep it. 


Dean 


Become Boring and Pigeonholed, 
Please 
SSS 
Hi. I’ve been a subscriber to LJ for several 
years, and I’ve never figured out your 
niche. Servers, or desktops? Sysadmins, 
application programmers, system develop- 
ers or home users? Your intention seems 
to be: appeal to everyone. Unfortunately 
for me, you cover so many different 

topics that there is very little in each issue 
for me. 


If you can’t figure out your niche, I'll let my 
subscription expire in January. 


jh 


When different areas of information technol- 
ogy can stop learning from each other, we'll 
pick a “niche”.—Ed. 


Ergonomic Comments on Ultimate 
Linux Box 
ay 
The case on the cover for your Ultimate 
Linux Box is gorgeous [August 2005]. 
Beautiful. Amazing. 


It also blows. It’s a terrible design. Ghastly. 
Ideal for a computer show, but awful to use. 
There’s no leg room on the box. For many 
folks, that would mean sitting obliquely or 
too far from the keyboard—both would 
cause strain. 
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Supermicro 5013G-MB 
* Intel® Pentium 4 Processor® at 3.0E GHz ¢ Includes CDROM, Floppy and Video 


e 1U Rackmount Chassis ° Lifetime toll free tech support 
¢ 512MB PC3200 DDR ¢ 3 Year Warranty 

* Maxtor 80GB Serial ATA Hard Drive 

* Dual Onboard Gigabit NIC’s $959 or lease for $33/mo. 


Or Promote Your Brand 


¢ Same Configuration as Above 

¢ Custom Branded With Your Logo 

¢ Worldwide Deployment and Support 
¢ Custom Branded Packaging Available 
¢ Configurations in 2U and 4U Available 
¢ Custom OS and Software Install 

¢ Custom Chassis Color Available 

¢ No Minimum Quantity Required 


$9 0 !) or lease for $38/mo. 


MBxX is the leader in custom appliances. Many premier application developers have chosen MBX as 
their manufacturing partner because of our experience, flexibility and accessibility. Visit our website or 
better yet, give us a call. Our phones are personally answered by experts ready to serve you. 


VJ www.mbx.com 
MOTHERBOARI 1.800.688.2347 


Intel, Intel Inside, Pentium and Xeon are trademarks and registered trademark of Intel Corporation or its subsidiaries in the United States and other countries. Lease calculated for 36 months, to approved business customers. Prices and 
specifications subject to change without notice. Setup fee may apply to certain branding options. Motherboard Express Company. 1101 Brown Street Wauconda, IL. 60084. 


Tactile response acts as a brake reducing the 
amount of impact on the end of the fingers 
(while, agreed, increasing the finger travel), 
so the keyboard used may actually increase 
hand stress for some. Its placement isn’t 
adjustable for height, and that can be catas- 
trophic for arm stress. 


On a much less important note, I’d point 
out that the case puts the cooling up very 
high indeed, into airspace often several 
degrees hotter than the rest of the room. 


Paul Pomerleau 


The coolant loops need to run well above the 
motherboard in order to get adequate con- 
vective cooling. For daily use, you can build 
a tall case without the monitor mounts or 
keyboard shelf.—Ed. 


i 


Another Happy Reader 
(Sa | 


Here’s a photo of my son Merit (about 26- 
months old) sitting on his trusty fire truck 
checking out my July 2005 issue of LJ. 
When he was done, he went back into the 
office and picked up an Advanced C 
Programming book! 


tim 


I had a great Father’s Day and hope all the 
other open-source dads did too. Of course, 
the best gift was Paige, my going-to-be- 
seven-in-September daughter—and you 
will see she painted Tux on a rock for me 
this Father’s Day. She always has been a 
good drawer, and her favorite program is 
Tux Paint with Tux Racer a close second. 


James M. Susanka 


Photo of the month gets you a one-year 
subscription or extension. Send photos to 
[jeditor @ ssc.com. 


Photo of the Month: a Linux Father’s Day 


Pipe Tip 
aT 
“Text Manipulation with sed” by Larry 
Richardson had some useful hints [July 
2005]. For instance, I wasn’t aware of the ! 
modification to the range field. 


But writing to a file at the same time you are 
reading it is decidedly dangerous. You are 
depending on the pipe buffer between cat and 
sed to hold the entire contents of the file. 
You are also banking on the assumption that 
cat will be started and allowed to fill its 
buffer before the file is written and, there- 
fore, truncated at zero length. A far better 
way to do this is in two steps: 


sed -e 's/$/ mycomputer/' < \ 
/etc/exports > tempfile \ 
&& mv tempfile /etc/exports 


The mv command is executed only if sed 
returns without problems. You don’t want to 
be overwriting important files with the 
wrong data! 


Allen Brown 


Her First Computer T-Shirt 
ae 
My daughter (2.2-years old) made the transi- 
tion. She was introduced to Potato Guy and 
Tux Paint on my Linux box. On this day, she 
learned everything she needed to know 
about the mouse operations. A very proud 
moment for me, she is on the road to becom- 
ing a geek just like her daddy. Elizabeth Su 
WOHID Certified (Wireless Optical Human 
Interface Device), T-shirt awarded shortly 
after. You can’t start them too young. This is 
only the beginning for them. I am just happy 
to be there. 


Jesse Apple 


LETTERS CONTINUED ON PAGE 94 
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1U Dual Opteron™ 4 SATA/SCSI 

High performance dual server for top-of-the-line 
processing power with ultra-dense storage 
capacity. 


¢ Dual AMD Opteron™ Processors w/HyperTransport 
and 1MB Cache 

¢ AMD 8000 Series Chipset w/64-bit Support 

¢ Up to 16GB DDR-400 Reg. ECC Memory 

¢ Up to 4 x 400GB (1.6TB) Hot-Swap SATA or 
4 x 300GB (1.2TB) Hot-Swap SCSI Drives 

¢ 400W AC Power Supply w/PFC 

¢ 5-Year Limited Warranty 


Starting at 7 | ,o95 


ABERDEEN STONEHAVEN A124 


1U Quad Opteron™ HPC 

64-bit HPC environment workhorse server/cluster 
node. Superior cooling with plenty of power to 
handle any project. 


* Quad AMD Opteron™ 800 Series Processors 
e AMD 8000 Series Chipset w/64-bit Support 
e Up to 32GB DDR-400 Reg. ECC Memory 

¢ Up to 2 x 300GB (600GB) SCSI Hard Drives 
© 500W Power Supply 

e Ultra Cool with Superb Air Flow 

¢ 5-Year Limited Warranty 


Quads Starting at $3,265 
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YOU CAN BUY THESE NOW 
OR WAIT TILL DELL FREEZES OVER 
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2U Dual Opteron™ 6 SATA/SCSI 
The highest performing 2U server available for the 
money. “Staggering ... Powerhouse Performance ... 
Highest Webbench numbers we've seen to date” — 
PC Magazine, December 27, 2004. 
¢ Dual AMD Opteron™ Processors w/HyperTransport 
and 1MB Cache 
¢ AMD 8000 Series Chipset w/64-bit Support 
e Up to 16GB DDR-400 Reg. ECC Memory 
e Up to 6 x 400GB (2.4TB) Hot-Swap SATA or 
6 x 300GB (1.8TB) Hot-Swap SCSI Drives 
¢ 460W Hot-Swap Redundant Power Supply 
¢ 5-Year Limited Warranty 


Starting at ‘2,8 75 
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2U Quad Opteron™ 3 SATA/SCSI 
Robust 64-bit server ideal for the HPC environment 
as a high performance server. Able to provide all the 
power and 1/0 for large databases and memory 
intensive projects. 
¢ Quad AMD Opteron™ Processors w/HyperTransport 
and 1MB Cache 
¢ AMD 8000 Series Chipset w/64-bit Support 
e Up to 32GB DDR-400 Reg. ECC Memory 
¢ Up to 3 x 400GB (1.2TB) Hot-Swap SATA or 
3 x 300GB (900GB) Hot-Swap SCSI Drives 
¢ 700W Power Supply 
e Ultra Cool with Superb Air Flow 
¢ 5-Year Limited Warranty 


Quads Starting at ‘3, | 25 
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ABERDEEN STONEHAVEN A381 


3U Dual Opteron™ 8 SATA/SCSI 
Gargantuan storage beast with a capacity of up to 
3.2TB, room for a dual-height tape drive, at an 
incomparable cost/TB ratio. 


¢ Dual AMD Opteron™ Processors w/HyperTransport 
and 1MB Cache 

e AMD 8000 Series Chipset w/64-bit Support 

e Up to 16GB DDR-400 Reg. ECC Memory 

¢ Up to 8 x 400GB (3.2TB) Hot-Swap SATA or 
8 x 300GB (2.4TB) Hot-Swap SCSI Drives 

¢ 760W Hot-Swap Redundant Power Supply 

¢ 5-Year Limited Warranty 


Starting at $2.9 75 


ABERDEEN STONEHAVEN A484 


4U Quad Opteron™ 8 SATA/SCSI 
Best of both worlds, all-inclusive server with 
enterprise-class 64-bit HPC Quad power along with 
maximum storage capacity. 


* Quad AMD Opteron™ Processors w/HyperTransport 
and 1MB Cache 

¢ AMD 8000 Series Chipset w/64-bit Support 

e Up to 32GB DDR-400 Reg. ECC Memory 

¢ Up to 8 x 400GB (3.2TB) Hot-Swap SATA or 
8 x 300GB (2.4TB) Hot-Swap SCSI Drives 

¢ 950W 3+1 Hot Swap Redundant Power Supply 

e Ultra Cool with Superb Air Flow 

¢ 5-Year Limited Warranty 


Quads Starting at $9,625 


888-297-7409 


www.aberdeeninc.com/linux 


a UPFRONT NEWS + FUN 


On the 


WEB 


To go along with this month’s 
theme of Personal Desktop, here 
are some articles from the Linux 
Journal Web site that will help 
you find your way through 
OpenOffice.org, try out some 
Linux audio software and rescue 
data from a hosed USB device: 


>> Do you want to move to 
OpenOffice.org but aren’t sure 
what to expect? Are you trying 
to convince friends and/or family 
members to give OOo a try, but 
they want to know about the 
learning curve? If so, Bruce 
Byfield’s article “OOo Off the 
Wall: What New Users Need to 
Know About OpenOffice.org” 
(www.linuxjournal.com/article/ 
8443) is suggested reading. 
Bruce sheds some light on OO0's 
“interface shortcomings” and 
“the limits of its on-line help”, 
as well as the “logic of its inter- 
face design and the importance 
of styles and templates in an 
efficient work flow”. 


>> Audio for Linux has come a long 
way in the past couple of years, 
and Dave Phillips continues his 


tour of what’s new for musicians 
and engineers, whether full-time 
or part-time. In recent months, he’s 


introduced us to FreeWheeling, “a 


powerful loop-based performance 


\" 


tool” (www.linuxjournal.com/ 
article/8445), as well as 
Qsynth and QJackCtl, GUI front 
ends that “make Linux audio 
tasks easier and faster, letting 
you get straight to the music” 
(www.linuxjournal.com/ 
article/8354). 


Finally, Collin Park shares his 
story of “How a Corrupted USB 
Drive Was Saved by GNU/Linux” 
(www.linuxjournal.com/article/ 
8366), offering hope to those of 
us who have lost important data 
and will lose it again. 


diff -u 


What’s New in Kernel Development 


After a long and difficult life, DevFS is final- 
ly being removed from the Linux kernel. 
Created by Richard Gooch, DevFS has been 
around for years, and it represented a serious 
attempt to cure the runaway /dev directory. 
Developing DevFS was an uphill battle 
against many detractors, but Richard did suc- 
ceed in creating a very useful tool. In the end, 
however, critics of DevFS won out, citing 
“unfixable races” and other problems, and 
Richard vanished from kernel development 
completely. Greg Kroah-Hartman and others 
then developed udev as a replacement for 
DevFS. Some lingering sense of the 2.6 kernel 
as a stable tree has made this decision slightly 
controversial even now, but almost certainly 
it’s not enough to influence the outcome. 
Farewell DevFS—it was a valiant effort. 

Recently, various folks have reported com- 
pilation problems when trying to compile the 
2.4 kernel with GCC version 4, and some 
developers have posted patches to address these 
issues; however, Marcelo Tosatti has stated that 
it is simply too late in the day for these sorts of 
patches to make it into the 2.4 tree. Unlike 2.6 
development, the maintainers of 2.4, 2.2 and 2.0 
have not decided to follow suit and abandon the 
idea that their trees must aim for stability. 
Marcelo has been trying to rein in 2.4 develop- 
ment ever since the first 2.6 kernel came out, 
but he has still allowed large IDE changes, new 
hardware support and other patches whose inva- 
siveness would typically fly in the face of a 
push for stability. And with 2.6 development 
showing no sign of slowing down, Marcelo has 
been under constant pressure to incorporate new 
features into 2.4 to be available to folks who 
needed 2.4’s stability. With the advent of the 
w.x.y.z tree, however, some of this pressure has 
undoubtedly flagged, and Marcelo has been 
able to tighten up the restrictions on what can 
and cannot get into 2.4 at this late date. 

The git versioning system continues to 
grow and strengthen. Andrew Morton’s 
-mm tree will be available as a git repository, 
although Andrew himself has no plans to use 
any versioning tool for actual development. 
The ALSA Project has migrated development 
to git, as has libata. Marcelo Tosatti’s 2.4 tree 
also will use git for ongoing development. 
Linus Torvalds is still very strongly involved 
with the project, and although mailing-list 
traffic has tapered off somewhat from its fran- 
tic early weeks, much of this is explained by 
the fact that folks now understand the basics 
of the tool, and the fundamental concepts no 


longer need to be explained to newcomers. 

In the midst of all the version-control 
upheaval, it’s hard to know for certain if the 
new w.x.y.z stable kernels are working out. But 
several kernel folks, including Jeff Garzik and 
Alan Cox, feel that this tree successfully pro- 
vides a stable kernel to supplement the 2.6 
tree’s ongoing large-scale development. Greg 
Kroah-Hartman and Chris Wright, the primary 
maintainers of the w.x.y.z tree, do seem to be 
doing a rigorous job, not only collecting and 
applying patches, but adhering to Linus 
Torvalds’ strict guidelines on what patches may 
be applied, and how and when they may be 
accepted. A number of aspects make this project 
less appealing than doing real development 
work, but Chris and Greg seem to be bearing up 
nicely, and the rest of us are the beneficiaries. 

Martin J. Bligh has put together a set of 
automatic testing scripts that compile and boot 
all official kernel releases (including the w.x.y.z 
kernels) and several prominent branches like the 
-mm tree, within 15 minutes of their release. If 
a kernel boots successfully, Martin’s scripts hit 
it with a variety of benchmarks. Compilation 
and boot results are recorded, benchmark results 
are graphed and everything is made available as 
a set of ongoing kernel.org Web pages. This is 
the sort of project that will not solve all bugs, 
but it will identify many trivial bugs, track per- 
formance problems across multiple kernel 
releases and may identify hard-to-find bugs that 
regular users would not normally see. 

The relatively recent introduction of 
Signed-Off-By tags in kernel patch submis- 
sions has made a huge difference in providing a 
trail of authorship, so that if anything like the 
SCO lawsuit occurs again, it will be easy to 
prove who wrote any disputed source code. 
This was, in fact, Linus Torvalds’ stated pur- 
pose in introducing the Signed-Off-By header. 
When first introduced, the idea was quite amor- 
phous, with few details settled. Since then, var- 
ious wrinkles have been introduced to improve 
its usefulness. One of the most recent of these 
is the addition of a From header as the first line 
of the body of patch e-mails. This header iden- 
tifies the true author of a given patch. Before 
this wrinkle, the true author was assumed to be 
the person with the bottom-most Signed-Off-By 
header. This, however, became confusing and 
was not always adhered to. The From header is 
intended to leave no doubt as to the original 
authorship of a given patch. 


—ZACK BROWN 


TOMOCTOBER 2005 WWW.LINUXJOURNAL.COM 


Cyclades A/terfath” OnSite makes 
branch office administration child's play 


exist Alberbadp manase™ > 
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The Next-Genera 


Cyclades AlterPath™ OnSite is the most cimpreherenel remote site and 
branch office administration appliance available. This small, inexpensive 
solution for controlling network equipment, servers and other IT infrastructure 
devices can 

¢ Access, diagnose and restore remote IT devices quickly 

© Download software to multiple devices automatically and simultaneously 

© Configure user information, system settings and operating parameters 


¢ Send alerts of intrusions, equipment failures and alarms 

copy of 

at i Managem oe The AlterPath OnSite combines the functionality of both serial console and KVM 
Be * white se ns over IP, allowing IT administrators to manage multiple servers and network 
ei devices through a single appliance. Cyclades brings it all together making 


remote site and branch office administration seem like child's play. 


Over 85% of Fortune 100 
choose Cyclades. 
www.cyclades.com/Ijb 


1.888.cyclades « sale yclades.com 
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UPFRONT NEWS + FUN 


Nigel McFarlane 


With the sudden death of Nigel 
McFarlane, the Web Development and 
Open Source Software communities, 
both in Australia and around the 
world, have lost one of their most 
well-known authors, consultants and 
pundits. 

Although in many ways a very pri- 
vate person, Nigel had a professional 
and personal network that spanned 


the globe and included such on-line 
luminaries as the lead engineer for the 
open-source browser Firefox Ben 
Goodger, and countless others in the 
Open Source, Web Development and 
Linux communities. Since his passing, 
many community sites, in a number of 
languages, have expressed their sor- 
row, a testament to Nigel’s influence. 

A real Melbourne boy, describing 
the city proudly as “the World’s most 
liveable”, Nigel had science degrees 
from both the University of 
Melbourne and LaTrobe University. 
Even when speaking in Sydney, he was 
always keen to get home as soon as 
possible, where he would bushwalk 
and ramble, swim and surf. 

Nigel forged a global reputation 
from his beloved Melbourne, in a way 
impossible until the 1990s. Many oth- 
ers have and will follow his lead, but as 
with much of what he did, here, too, 
Nigel was a pioneer. Since 1997, Nigel 
had become well known and respect- 
ed in the Web Development and more 
recently Open Source Technology com- 
munities through the publication of 
several successful books on JavaScript, 
Mozilla and most recently, the increas- 
ingly popular free open-source brows- 
er Firefox. 


Two earlier books on JavaScript, 
Instant JavaScript in 1997 and the co- 
authored Professional JavaScript in 2001, 
are still considered by many to be 
among the best books on the subject. 
More recently, the benchmark Rapid 
Application Development with Mozilla, and 
Firefox Hacks carved out a place in the 
increasingly important Open Source 
community. 

Nigel’s writing extended to the 
columns “Searching for Substance” for 
InformIT, and articles for such publica- 
tions as Linux Journal, DevX, Builder.com, 
CNet, The Age and the Sydney Morning 
Herald. Nigel was an entertaining speak- 
er as well as a writer. | particularly recall 
chairing a conference session that Nigel 
presented late last year. Often confer- 
ence-goers are anxious to get early 
places in the meal queue, but although 
we had gone overtime for lunch, Nigel 
captivated the room. When offered the 
opportunity to break, the entire room 
turned it down, glued as they were to 
Nigel's presentation. 

Generous with his time, energies 
and knowledge, Nigel contributed to 
mailing lists, newsgroups and forums, 
as well as speaking to audiences large 
and small at conferences and for user 
groups. His reach went far beyond 


HLA Adventure 


members.tripod.com/~panks/hlaadv.html 


When Zork appeared on the scene in 
the late 1970s, computer enthusiasts 
from around the world were instantly 
hooked on the interactive fiction 
genre known fondly as the Text 
Adventure game. 

HLA Adventure is the latest in a 
long line of public domain and free 
software text adventures being 
released by people all over the world. 
It combines elements from MUDs, 
Advanced Dungeons & Dragons and 
J.R.R. Tolkien’s famous 
The Lord of the Rings. 

Using verbs and nouns to communi- 
cate with the game world, the player 
moves about HLA Adventure with but a 
simple goal in mind: slay the menacing 
dragon at the end of a large expanse 
of caves. While solving this main quest, 


warp ire 
dtagon 


The dragon bellows,’I am the last of my kind. find here you are, a 
dirty human, come to slay me?! I breathe fire upon trees, rustling 
the bad kami out of the poison forests and valleys of this forgotten 
world. I keep my peace inside these caves, only to be disturbed by 
the wretched humans which pollute this landscape! I was once happy 
in my silence here, yet you come here to make me miserable! I won’t 


allow it, KRnave! Prepare to feel my wrath!!!’ 


The dragon moves about angrily, stomping the floor! 


7 
fi large dragon with tough scales and powerful claws. 
> 


You are carrying: 
lantern 
rope 
key 
sword 
longsword 
twreided) 


Cworn) 


the player is also presented with nine 
other unique quests, which allow the 
player to find items and equip 
weapons, armor and a brightly lit 
lantern. Even a magical flute plays a 
role—useful in putting magical beasts 
to sleep. 

Players will encounter hellhounds, 
werewolves, vampires, hobbits, ghosts, 


barbarians and demigorgons. Talk to 
creatures in the game with the TALK 
TO command. Once you have acquired 
the necessary armament and passed 
the requisite number of quests, you 
can then enter into the cave and slay 
the dragon for good. 

Despite some bugs in the game, 
HLA Adventure is a solid, robust open- 
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Reclaim lost time 
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The world’s first Linux management appliance 


Levanta Intrepid ™ 


Plug the Levanta Intrepid ™ into your network and perform the most important 
Linux management tasks in a fraction of the time you spend now. And gain 


power and flexibility that you’ve never had before: 


== Fast & Portable: Provision servers or workstations practically 


anywhere, anytime — in minutes. Swap them around, mix it up. 30-Day 

== Flexible: Supports commodity hardware, blades, virtual machines, Money-Back Guarantee 
and even mainframes. 5 

= Out of the Box: Includes pre-defined templates for servers, Order online by 10/30/05 
workstations, & software stacks. Or create your own. Get $500 Off 

= Total Control: Track any file changes, by any means, at any time. Enter PROMO CODE: LJ1005 


And undo them at will. 


== Disaster Recovery: Bring dead machines quickly back to life, 
even if they’re unbootable. 


Based upon technology that’s already been proven in Fortune 500 
enterprise data centers. Now available in a box, priced for smaller aw 


environments. Just plug it in and go. 


LEVANTA 
www.levanta.com 
© 2005 Levanta, Inc. All rights reserved. Levanta and the Levanta logo are registered marks of Levanta, Inc 1.877.LEVANTA 


TOOLBOX AT THE FORGE 


Ruby on 
Rails 


Explore a Web development framework that comes 
with its own Web server, magically keeps track of 
details for you and integrates new code without 
restarting. BY REUVEN M. LERNER 


uby, an interpreted programming language that looks 

and feels like a cross between Smalltalk and Perl, 

has been around for about ten years. Ruby has been 

gaining in popularity over the last few years, partly 
because of the release of English-language books and docu- 
mentation. In addition, programmers have become more inter- 
ested in finding an alternative to Perl and Python for their gen- 
eral-purpose programming needs. 

Ruby’s popularity might have continued to grow slowly 
were it not for Ruby on Rails, a Web development framework 
that has become the focus of enormous attention. Everyone in 
the Web development world seems to be talking about Rails; 
magazine articles, blog postings, conference tracks and even 
some new books all are dedicated to Rails. Rails is supposed to 
be elegant, easy to use and easy to modify. Even developers 
with no previous Ruby experience are switching to Rails. 

Does Rails live up to the hype surrounding it? To a large 
degree, I believe the answer is “yes’”—it has a relatively shal- 
low learning curve, it connects easily and quickly to relational 
databases and it makes the creation of many small- and medi- 
um-sized sites faster and easier than I would have expected. 
But, of course, no framework is perfect, particularly one that 
was released publicly only one year ago. It remains to be 
seen whether Rails can hold up against more-established 
technologies on several different fronts. 

This month, we begin to look at several aspects of Ruby on 
Rails, so you can decide for yourself if my assessment is accurate. 
We begin by installing and configuring a basic Rails application. 
Over the next few installments of At the Forge, we will extend our 
application in several different ways, considering the ways in 
which Rails allows us to create and modify our applications. 


Installing Rails 
The first step in creating a Rails application is to install Ruby 
and then Rails itself. Most modern Linux distributions come 
with Ruby, although only the latest released version as of this 
writing (1.8.2) works with the most recent version of Rails 
(0.12.1). New versions of Rails have been coming out fre- 
quently, which means that one or both of these versions might 
have changed by the time you read this. 

Assuming you have installed Ruby, you next need to install 
Gems. It provides access to the Ruby Gems library, which is 
something of a cross between SourceForge and Perl’s CPAN 


(see the on-line Resources). Download and unpack the most 
recent .tar.gz file: 


tar -zxvf rubygems-0.8.10.tar.gz 
Enter the directory as the root user and type: 
ruby setup.rb all 


This installs the entire Gems package. Among other 
things, this installs the gem program in /usr/bin. You then 
can install Rails, which is distributed via Gems, with the 
following command: 


gem install --remote rails 


As with such systems as CPAN and Debian’s apt, the gem 
program is smart enough to identify and download any depen- 
dencies it might encounter. By default, you need to answer “y” 
explicitly when asked if you are interested in installing any 
dependencies. Because Rails depends on a number of other 
packages, you should be sure to answer “‘y” when prompted. 

When you are returned to the shell prompt, you can assume 
that Rails has been installed. However, this is not quite enough. 
If you are interested in working with a relational database, 
you also need to install a database interface library. Because 
I work with PostgreSQL, I installed the pure Ruby client, 


called postgres-pr: 
gem install --remote postgres-pr 


Somewhat confusingly, there also is a set of PostgreSQL 
client libraries (called postgresql) that can be used with Ruby. 
However, it seems as though most Rails developers are work- 
ing with the postgres-pr library, at least for now. 


Creating an Application 

Once Rails is installed, we can create a simple “Hello, world” 
program. To do this, we use the rails command, which is 
installed in /usr/bin/ by default. Because our example applica- 
tion is a Weblog, we call the application blog. For reference, 
the name of the application doesn’t have to be linked to the 
name of the URL under which it will appear. Type: 


rails blog 


Running this produces a fair amount of output, listing the 
files that have been created on our filesystem. When we give 
only a single name, blog, the application is created inside of a 
directory with that name. We can keep all of our applications 
inside of a single container directory, such as ~/Rails, with: 


mkdir ~/Rails 
rails ~/Rails/blog 


If we look inside the newly created application directory, 
we see a number of directories and files. The script directory 
contains administrative programs, written in Ruby, of course. 
The public directory contains static HTML files, as well as 
images, stylesheets, JavaScript code and templates that you 
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EmperorLinux 


...where Linux & laptops converge 


You choose your laptop... from a wide selection of top tier laptops manufactured by 
IBM/Lenovo, Dell, Sharp, and Sony. They come in all sizes from two pound ultra-portables to eight 
pound desktop replacements; get exactly as much Linux laptop as you need. Need help deciding? 


© Sharp Actius MM20/MP30 © Sony VAIO S380 
© 10.4" XGA screen © 13.3" WXGA+ screen 
© 1.6 GHz Transmeta Efficeon *® X@1280x800 


© 20-40 GB hard drive 

@ 512-1024 MB RAM 

¢ CDRW/DVD (MP30) 

© 802.11b/g wireless 

* ACPI hibernate 

e1" thin 

¢ Ask about the 3D Molecule 


© 1.6-2.13 GHz Pentium-M 
© 40-100 GB hard drive 

© 256-1024 MB RAM 

® CDRW/DVD or DVD-RW 

© 802.11b/g wireless 

e ACPI hibernate 

e Ask about the 17" Gazelle 


You choose your distribution... from among the most popular Linux distributions 
available. We'll install the distribution you select, then we'll install our custom, laptop-specific kernel 
and configure your distribution for full hardware support, including: X at the native resolution, 
wireless ethernet, power management, 3-D graphics, optical drives, and more. 


The Toucan: 5 lb Linux The Rhino: 7 lb Linux 


© |BM/Lenovo ThinkPad T series 
© 14.1" SXGA+/15.0"' UXGA 

© X@1400x1050/X@1600x1200 
° ATl FireGL graphics 

© 1.6-2.13 GHz Pentium-M 7xx 

© 40-80 GB hard drive 

e 512-2048 MB RAM 

© CDRW/DVD or DVD-RW 

© APM suspend/hibernate 

° Ask about the 3 lb Raven X41 


* Dell Latitude D810/M70 

© 15.4" WUXGA screen 

® X@1920x1200 

® NVidia Quadro or ATI Radeon 
© 1.73-2.26 GHz Pentium-M 7xx 
* 30-100 GB hard drive (7200 rpm) 
® 256-2048 MB RAM 

e CDRW/DVD or DVD+RW 

* 802.11a/b/g wireless, GigE 
¢ Ask about the tiny Koala X1 


Let EmperorLinux do the rest. since 1999, EmperorLinux has provided pre-installed 


LM ar eon cireeR Linux laptop solutions to universities, corporations, and individual Linux enthusiasts. We specialize 
des Socio F | in the installation and configuration of the Linux operating system on a wide range of the finest laptop 
oy RC aed en and notebook computers made by IBM/Lenovo, Dell, Sharp, and Sony. We offer a range of the latest 
lh satan Linux distributions, as well as Windows dual boot options. All systems come with one year of Linux 


technical support by both phone and email, and full manufacturers’ warranties apply. 


www.EmperorLinux.com 1-888-651-6686 


Model prices, specifications, and availability may vary. All trademarks are the property of their respective owners. 


You 


HIGH PERFORMANCE 
COMPUTING SOLUTION 
HAS ARRIVED. 


VXRACK™ with the Intel® Xeon™ processor 
helps you simplify computing operations, 
accelerate performance and 

accomplish more in less time. 


Choose one of the 3 
convenient rack sizes 


VXR-96 

Rack accomodating up to 

96 VXBlades/192 Processors 
36TB of aggregated Storage 
1.15TB of Global Memory 
Power Distribution Included 
Patented Architecture 

Advanced Cooling System 
Integrated InfiniBand Cable Mgnt. 


VXR-128 $ 1,750.00° 
Rack accomodating up to 
128 VXBlades/256 Processors 
48TB of aggregated Storage 
1.5TB of Global Memory VXR-72 
Power Distribution Included Rack accomodating up to 
Patented Architecture 72 VXBlades/144 Processors 
Advanced Cooling System 27TB of aggregated Storage 


864GB of Global Memory 
Power Distribution Included 
Patented Architecture 

Advanced Cooling System 
Integrated InfiniBand Cable Mgnt. 


$ 1,590.00° 


Integrated InfiniBand Cable Mgnt. 


$ 2,190.00° 


For more Information call 
or visit us at 


TECHNOLOGIES 


Intel, Intel logo, Intel Inside, Intel Inside Jogo, Intel Centrino, Intel Centrino log 
“Important Information. All prices, specifications and promotional offer 


VXB-7221B 

Intel SE7221B Motherboard 
800MHz Front Side Bus 

Intel® Pentium® 4 3.2GHz 

1GB DDR2 400 Memory 

Single 40GB 7200RPM ATA Drive 
One PCI/Express Slot Available 
Dual 10/100/1000 Intel Lan Port 
350W Power Supply 


$ 985.00 


VXB-7501W 

Intel SE7501W Motherboard 
533MHz Front Side Bus 

2 x Intel® Xeon™ 3,06GHz 
2GB DDR 333 ECC Reg.Mem 
Single 40GB 7200RPM ATA Drive 
One PCI/X Slot Available 

Dual 10/100/1000 Intel Lan Port 
350W Power Supply 


$ 2,355.00 


VXB-7520J 

Intel SE7520J Motherboard 
800MHz Front Side Bus 

2 x Intel® EM64T Xeon™ 3.2GHz 
2GB DDR2 400 ECC Reg.Mem 
Single 40GB 7200RPM ATA Drive 
One PCI/Express Slot Available 
Dual 10/100/1000 Intel Lan Port 
500W Power Supply 


$ 2,950.00 


Choose one or more 
type of VXBlade 


: For example you choose the following: One VXR-96 with 
5, 
3) ihe Liebe 45 Dual Intel® EM64T Xeon™ and 40 Single Intel® Pentium®. 
You take 1 (VXR-96) + 48 (VXB-7520J) + 40 (VXB-7221B)...That’s it 
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THE FUTURE OF CLUSTER TECHNOLOGY 


Easy as 1, 2, 3... 


CIARA TECHNOLOGIES...A GLOBAL SOLUTION PROVIDER. 

Ciara Technologies is a world-class computer systems manufacturer. Ciara designs, develops, 
manufactures, markets, services, and supports a variety of computer systems including graphic 
workstations, rackmount and tower servers, networked storage and the newly acclaimed VXRACK™ 
Cluster Technology. The company's state of the art supercomputer cluster is based on the Inte! |AS2 
and IA64 architectures and utilizes Linux operating systems. We are proud to be recognized by Intel as 
an "Intel Premier Provider". Choosing Ciara is choosing a single point of contact for all your IT 
requirements. All our products are built under the ISO 9001 standards and regulations. The growth of 
Ciara enabled the company to move its 300+ employees,in February 2003, to an ultra-modern plant of 
576,000 saft.. Ciara now has the capability of producing more than 500,000 units per year. 


866-7VX-RACK (866-789-7225) 
WWW.VXRACK.COM 


| Pentium ill Xeon are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States and other counties 
a for typography errors, Photographies errors, pricing errors, All pricing NUS dollar. Shipping and applicable taxes are not included. 
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may use in your application. 

The directory you are mostly likely to work with is app, 
which contains the application itself. The app directory 
contains subdirectories named models, views and con- 
trollers. This design reflects the fact that Rails uses the 
MVC (model/view/controller) style widely used in many 
modern desktop and Web applications. 

In an MVC architecture, we divide our work into three 
parts—the controller, which acts like a switchboard, invok- 
ing the appropriate model and view; the model, which con- 
tains the data and some of the logic; and the view, which 
displays information to the user. If you have ever built a 
database-backed site with PHP and Smarty templates or 
with Zope and its Page Templates or even with Java and 
JavaServer Pages (JSPs), you already are familiar with at 
least some of these ideas. Rails simply makes them more 
explicit with its prenamed directory structure. 

Although it can’t do much, we now can start our empty 
Rails application with: 


cd ~/Rails/blog 
ruby script/server 


This starts the WEBrick HTTP server on port 3000. To 
access this fairly empty Rails site, we point our browsers to 
an appropriate IP address or hostname. In my particular 
case, I started Rails on my test server, whose IP address 
is 192.168.2.3. I thus point my Web browser to 
http://192.168.2.3:3000/. And sure enough, there I see a 
“Welcome on board” message, indicating I have set up 
Rails correctly. 


Customizing the Behavior 

Now that we know how to see the default message, let’s move 
toward a “Hello, world” program. In Rails, there are two basic 
ways to do this. We can create a controller that returns HTML 
to the user’s browser, or we can create a view that does the 
same. Let’s try it both ways, so that we can better understand 
the relationship between controllers and views. 

If all we want to do is include a simple, static HTML docu- 
ment, we can do so in the public directory. In other words, the 
file blog/public/foo.html is available under WEBrick—started 
by executing blog/script/server—at the URL /foo.html. 

Of course, we’re interested in doing something a bit more 
interesting than serving static HTML documents. We can do 
that by creating a controller class and then defining a method 
within that class to produce a basic “Hello, world” message. 
Admittedly, this is a violation of the MVC separation that Rails 
tries to enforce, but as a simple indication of how things work, 
it seems like a good next step. 

To generate a new controller class named MyBlog, we 
enter the blog directory and type: 


ruby script/generate controller MyBlog 


Each time we want to create a new component in our Rails 
application, we call upon script/generate to create a skele- 
ton. We then can modify that skeleton to suit our specific 
needs. As always, Rails tells us what it is doing as it creates the 
files and directories: 


exists app/controllers/ 
exists app/helpers/ 
create app/views/my_blog 


test/functional/ 

app/controllers/my_blog controller.rb 
test/functional/my_blog controller_test.rb 
app/helpers/my_blog helper.rb 


exists 
create 
create 
create 


Also notice how our controller class name, MyBlog, has 
been turned into various Ruby filenames, such as 
app/views/my_blog and app/helpers/my_blog_helper.rb. Create 
several more controller classes, and you should see that all of 
the names, like FooBar, are implemented in files with names 
like foo_bar. This is part of the Rails convention of keeping 
names consistent. This consistency makes it possible for Rails 
to take care of many items almost magically, especially—as we 
will see next month—when it comes to databases. 

The controller that interests us is my_blog_controller.rb. If 
you open it up in an editor, you should see that it consists of 
two lines: 


class MyBlogController < ApplicationController 
end 


In other words, this file defines MyBlogController, a class 
that inherits from the ApplicationController class. As it stands, 
the definition is empty, which means that we have neither 
overridden any methods from the parent class nor written any 
new methods of our own. Let’s change that, using the built-in 
render_text method to produce some output: 


class MyBlogController < ApplicationController 


def hello_world 
render_text "Hello, world" 
end 


end 


After adding this method definition, we can see its results 
by going to http://192.168.2.3:3000/MyBlog/hello_world. 

Notice how the URL has changed: static items in the 
public directory, such as our file foo.html, sit just beneath the 
root URL, /. By contrast, our method hello_world is accessed 
by name, under the controller class that we generated. Also 
notice that we did not need to restart Rails in order to create 
and test this definition. As soon as a method is created or 
changed, it immediately is noticed and integrated into the 
current Rails system. 

If we define an index method for our controller class, we 
can indicate what should be displayed by default: 


class MyBlogController < ApplicationController 
def hello_world 
render_text "Hello, world" 


end 


def index 
render_text "I am the index!" 
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end 
end 


Of course, it’s not that exciting to be able to produce static 
text. Therefore, let’s modify our index method such that it uses 
Ruby’s built-in Time object to show the current date and time: 


def index 
render_text "The time is now " + Time.now.to_s + "\n" 
end 


And voila! As soon as we save this modification to disk, 
the default URL (http://192.168.2.3:3000/MyBlog/, on my 
computer) displays the time and date at which the request was 
made, as opposed to a never-changing “Hello, world” message. 

Let’s conclude this introduction to Rails by separating the 
controller from its view once again. In other words, we want to 
have the controller handle the logic and the view handle the 
HTML output. Once again, Rails allows us to do this easily by 
taking advantage of its naming conventions. For example, let 
us modify our index method again, this time removing its 
entire body: 


def index 
end 


This might seem strange at first glance. It tells Rails that 
the MyBlog controller class has an index method. But it does- 
n’t generate any output. If you attempt to retrieve the same 
URL as before, Rails produces an error message indicating that 
it could not find an appropriate template. 

Because the template is a view, we can define it inside of 
the blog/app/views directory of our application. And because 
we are defining the index view for the MyBlog class, we modi- 
fy the index.rhtml file in the my_blog subdirectory of views. 
Notice how Rails turns ThisName into this_name when it 
comes to directories. Doing so saves users from having to 
think about capitalization in URLs, while staying consistent 
with traditional Ruby class naming conventions. 

.thtml files are a Ruby version of the same kind of template 
that you might have seen before. It acts similarly to ASP and 
JSP syntax, with <% %> blocks containing code and <%= %> 
blocks containing expressions that should be interpolated into 
the template. However, nothing stops us from creating an 
.thtml template that actually is static: 


<html> 
<head> 
<title> 
Hello, again! 
</title> 
</head> 


<body> 
<p>Hello, again!</p> 


</body> 


</html> 


Consider what happens now if you attempt to load 
MyBlog in your browser. The controller class MyBlog is 
handed the request. Because no method was named explicitly, 
the index method is invoked. And because index doesn’t 
produce any output, the my_blog/index.rhtm] template is 
returned to the user. 

Finally, let’s take advantage of our template’s dynamic 
properties to set a value in the controller and pass that along to 
the template. We modify our index method to read: 


def index 
@now = Time.now.to_s 
end 


Notice how we have used an @ character at the beginning 
of the variable @now. I found this to be a little confusing at 
first, as @ normally is used as a prefix for instance variables 
in Ruby. But it becomes fairly natural and logical after a 
little time. 

Finally, we modify our template such that it incorporates 
the string value contained in @now: 


<html> 
<head> 
<title> 
Hello, world! 
</title> 
</head> 


<body> 

<p>Hello, world!</p> 

<p>It is now <%= @now %>.</p> 
</body> 


</html> 


Once again, you can retrieve the page even without restart- 
ing Ruby. You should see the date and time as kept on the serv- 
er, updated each time you refresh the page. 


Conclusion 
Ruby on Rails is, without a doubt, one of the most talked-about 
Web technologies to emerge in the past few years. This month, 
we saw how straightforward it is to create a new Rails applica- 
tion, to create a controller and a view and to integrate them 
using a combination of naming conventions and relatively stan- 
dard template syntax. However, we did not discuss views, par- 
ticularly those associated with a relational database. Next 
month, we will do exactly that, connecting Rails to the 
PostgreSQL database. I believe doing so will begin to show 
why people are so excited about Rails and why it might be a 
good tool for many Web developers to learn. 

Resources for this article: www.linuxjournal.com/article/ 
8457.8 


Reuven M. Lerner, a longtime Web/database con- 
sultant and developer, now is a graduate student in 
the Learning Sciences program at Northwestern 
University. His Weblog is at altneuland.lerner.co. il, 
and you can reach him at reuven@lerner.co.il. 
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Network 


Programming in 
the Kernel 


Take a tour of the kernel’s networking functionality by 
writing a network client that runs in kernel space. 
BY PRADEEP PADALA AND RAVI PARIMI 


Il Linux distributions provide a 

wide range of network appli- 

cations—from demons that 

provide a variety of services 
such as WWW, mail and SSH to client 
programs that access one or more of these 
services. These programs are written in 
user mode and use the system calls pro- 
vided by the kernel to perform various 
operations like network read and write. 
Although this is the traditional method of 
writing programs, there is another inter- 
esting way to develop these applications 
by implementing them in the kernel. The 
TUX Web server is a good example of an 
application that runs inside the kernel and 
serves static content. In this article, we 
explain the basics of writing network 
applications within the kernel and their 
advantages and disadvantages. As an 
example, we explain the implementation 
of an in-kernel FTP client. 


Advantages and Disadvantages of 
In-Kernel Implementations 

Why would one want to implement 
applications within the kernel? Here are 
a few advantages: 


m When a user-space program makes a 
system call, there is some overhead 
associated in the user-space/kernel- 
space transition. By programming all 
functionality in the kernel, we can 
make gains in performance. 


m™ The data corresponding to any appli- 
cation that sends or receives packets 
is copied from user mode to kernel 
mode and vice versa. By implement- 
ing network applications within the 
kernel, it is possible to reduce such 
overhead and increase efficiency by 


not copying data to user mode. 


™@ In specific research and high-perfor- 
mance computing environments, there 
is a need for achieving data transfers 
at great speeds. Kernel applications 
are useful in such situations. 


On the other hand, in-kernel imple- 
mentations have certain disadvantages: 


™ Security is a primary concern within 
the kernel, and a large class of user- 
mode applications are not suitable to 
be run directly in the kernel. 
Consequently, special care needs to 
be taken while designing in-kernel 
applications. For example, reading 
and writing to files within the kernel 
is usually a bad idea, but most appli- 
cations require some kind of file I/O. 


™ Large applications cannot be imple- 
mented in the kernel due to memory 
constraints. 


Network Programming Basics 
Network programming is usually done 
with sockets. A socket serves as a com- 
munication end point between two pro- 
cesses. In this article, we describe net- 
work programming with TCP/IP sockets. 
Server programs create sockets, bind 
to well-known ports, listen and accept 
connections from clients. Servers are usu- 
ally designed to accept multiple connec- 
tions from clients—they either fork a new 
process to serve each client request (con- 
current servers) or completely serve one 
request before accepting more connec- 
tions (iterative servers). Client programs, 
on the other hand, create sockets to con- 
nect to servers and exchange information. 


FTP Client-Server Interaction 

Let’s take a quick look at how an FTP 
client and server are implemented in user 
mode. We discuss only active FTP in this 
article. The differences between active 
and passive FTP are not relevant to our 
discussion of network programming here. 


Socket Programming Basics 

Here is a brief explanation of the design 
of an FTP client and server. The server 
program creates a socket using the 
socket() system call. It then binds on a 
well-known port using bind() and 
waits for connections from clients using 
the Listen() system call. The server 
then accepts incoming requests from 
clients using accept() and forks a new 
process (or thread) to serve each incom- 
ing client request. 

The client program creates a control 
socket using socket () and next calls 
connect() to establish a connection with 
the server. It then creates a separate 
socket for data transfer using socket () 
and binds to an unprivileged port 


client server 


socket() 


) 


connect( 


Figure 1. The FTP protocol uses two sockets: one 


for control messages and one for data. Here’s how 


the first connection, used for commands, gets set up. 
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(>1024) using bind(). The client now listen()s on this port 
for data transfer from the server. The server now has enough 
knowledge to honor a data transfer request from the client. 
Finally, the client uses accept () to accept connections from the 
server to send and receive data. For sending and receiving data, 
the client and server use the write() and read() or sendmsg() 
and recvmsg() system calls. The client issues close() on all 
open sockets to tear down its connection to the server. Figure | 
sums it up. 


FTP Commands 

Here is a list of a few FTP commands we used. Because our 
program provides only a basic implementation of the protocol, 
we discuss only the relevant commands: 


™ The client sends a USER <username>\r\n command to the 
server to begin the authentication process. 


™ To send the password, the client uses PASS password\r\n'. 


@ In some cases, the client sends a PORT command to inform 
the server of its preferred port for data transfer. In such 
cases, the client sends PORT <al,a2,a3,a4,p1,p2>\r\n. 
The RFC for FTP requires that the al—a4 constitute the 32- 
bit IP address of the client, and p1—p2 constitute the 16-bit 
port number. For example, if the client’s IP address is 
10.10.1.2 and it chooses port 12001 for data transfer, the 
client sends PORT 10,10,1,2,46,225. 


m™ Some FTP clients request, by default, that data be trans- 
ferred in binary format, while others explicitly ask the serv- 
er to enable data transfer in binary mode. Such clients send 


USER parimi\r\n 
331 Password required for parimi\r\n 


PASS foobar\r\n 


230 User parimi logged in\r\n 


PORT a1,a2,a3,a4,p1,p2\r\n 


200 PORT command successful 


TYPE I\r\n 
200 TYPE set to | 


Figure 2. The client issues FTP commands over the control connection to set up 


the file transfer. 


a TYPE I\r\n command to the server to request this. 


Figure 2 is a diagram that shows a few FTP commands and 
their responses from the server. 


Socket Programming in the Kernel 
Writing programs in the kernel is different from doing the 
same in user space. 

We explain a few issues concerned with writing a network 
application in the kernel. Refer to Greg Kroah-Hartman’s arti- 
cle “Things You Never Should Do in the Kernel” (see the on- 
line Resources). First, let’s examine how a system call in user 
space completes its task. For example, look at the socket () 
system call: 


sockfd = socket (AF_INET,SOCK_STREAM,90) ; 


When a program executes a system call, it traps into the 
kernel via an interrupt and hands over control to the kernel. 
Among other things, the kernel performs various tasks, such as 
saving contents of registers, making changes to address space 
boundaries and checking for errors with system call parame- 
ters. Eventually, the sys_socket() function in the kernel is 
responsible for creating the socket of a specified address and 
family type, finding an unused file descriptor and returning this 
number back to user space. Browsing through the kernel’s code, 
we can trace the path followed by this function (Figure 3). 


User Space Kernel Space 


as eee 


socket() ———>| 1. Save registers 

2. Change address 
space boundaries 

3. Check system call 
parameters for errors 

4. Miscellaneous other 
checks 


1. sock_create() 


[t——> sys_socket() —> 2. sock_map_fd() 


3. Get socket descriptor 


Copy socket descriptor to user-space 
—< 


Figure 3. Behind the scenes of a system call: when user space executes socket(), 


the kernel does necessary housekeeping and then returns a new file descriptor. 


Design of an FTP Client 

We now explain the design and implementation of a kernel 
FTP client. Please follow through the code available at the 
Linux Journal FTP site (see Resources) as you read through 
this article. The main functionality of this client is written in 
the form of a kernel module that adds a system call dynamically 
that user-space programs can invoke to start the FTP client pro- 
cess. The module allows only the root user to read a file using 
FTP. The user-space program that calls the system call in this 
module should be used with extreme caution. For example, it is 
easy to imagine the catastrophic results when root runs: 


./a.out 10.0.0.1 10.0.0.2 foo_file /dev/hdal1/* 
and overwrites /dev/hdal with a downloaded file from 10.0.0.1. 
Exporting sys_call_table 


We first need to configure the Linux kernel to allow us to add 
new system calls via a kernel module dynamically. Starting 


Z4H OCTOBER 2005 WWW.LINUXJOURNAL.COM 


L Series Laptop - LS1250-L 
Light &Thin Performance 
Starting at $1,562.54 


G Series Laptop - GW1550-L 
Essential Technology on a Budget 
Starting at $1,302.54 


X Series Laptop - XW1550-L 
Extreme Technology & Performance 
Starting at $1,608.04 


R Cubed Technologies has provided 
pre-installed Linux laptops without OS 
tax since 2003. We customize the 
Fedora Linux distribution for each 
laptops configuration providing support 
for: PCMCIA, USB, FireWire, X, 
CD/DVD/CDRW/DVDRW, Sound, Power 
Management, Ethernet, Modem, 
Wireless, and more. Our laptops are 
equipped with Intel Centrino Mobile 
Technology. We also offer Windows dual 
boot options. All of our laptops come 
with a one year parts and labor warranty. 
Visit us online at www.shoprcubed.com 
or call 309.34.CUBED for details. 


3 
| P Technologies* 


Working Hard to Bring Technologies to Life. 


309.34.CUBED 
www.shoprcubed.com 


DON'T BE SQUARE! 


CET CUBED! 


All models, prices, and availability may 
©Copyright 2005 


their respective owners. 


TOOLBOX KERNEL KORNER 


with version 2.6, the symbol sys_call_table is no longer 
exported by the kernel. For our module to be able to add a sys- 
tem call dynamically, we need to add the following lines to 
arch/i386/kernel/i386_ksyms.c in the kernel source (assuming 
you are using a Pentium-class machine): 


extern void *sys_call_table; 
EXPORT_SYMBOL(sys_call_table) ; 


After recompiling the kernel and booting the machine into it, 
we are all set to run the FTP client. Refer to the Kernel Rebuild 
HOWTO (see Resources) for details on compiling a kernel. 


Module Basics 

Let’s examine the code for the module first. In the code snip- 
pets in this article, we omit error checking and other irrelevant 
details for clarity. The complete code is available from the LJ 
FTP site (see Resources): 


#include <linux/init.h> 
#include <linux/module.h> 
#include <linux/kernel.h> 


/* For socket etc */ 
#include <linux/net.h> 
#include <net/sock.h> 
#include <linux/tcp.h> 
#include <linux/in.h> 
#include <asm/uaccess.h> 
#include <linux/file.h> 
#include <linux/socket.h> 
#include <linux/smp_lock.h> 
#include <linux/slab.h> 


int ftp_init(void) 


{ 
printk(KERN_INFO FTP_STRING 
"Starting ftp client module\n"); 
sys_call_table[SYSCALL_NUM] = my_sys_call; 
return 0; 
} 
void ftp_exit(void) 
{ 
printk(KERN_INFO FTP_STRING 
"Cleaning up ftp client module, bye !\n"); 
sys_call_table[SYSCALL_NUM] = sys_ni_syscall; 
} 


The program begins with the customary include directives. 
Notable among the header files are linux/kernel.h for 
KERN_ALERT and linux/slab.h, which contains definitions for 
kmalloc() and linux/smp_lock.h that define kernel-locking rou- 
tines. System calls are handled in the kernel by functions with 
the same names in user space but are prefixed with sys_. For 
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example, the sys_socket function in the kernel handles the 
task of the socket () system call. In this module, we are using 
system call number 223 for our new system call. This method 
is not foolproof and will not work on SMP machines. Upon 
unloading the module, we unregister our system call. 


The System Call 

The workhorse of the module is the new system call that per- 
forms an FTP read. The system call takes a structure as a 
parameter. The structure is self-explanatory and is given below: 


struct params { 

/* Destination IP address */ 

unsigned char destip[4]; 

/* Source IP address */ 

unsigned char srcip[4]; 

/* Source file - file to be downloaded from 
the server */ 

char src[64]; 

/* Destination file - local file where the 
downloaded file is copied */ 

char dst[64]; 

char user[16]; /* Username */ 

char pass[64]; /* Password */ 


The system call is given below. We explain the relevant 
details in next few paragraphs: 


asmlinkage int my_sys_call 

(struct params __user *pm) 

{ 
struct sockaddr_in saddr, daddr; 
struct socket *control= NULL; 
struct socket *data = NULL; 
struct socket *new_sock = NULL; 


int r = -1; 
char *response = kmalloc(SNDBUF, GFP_KERNEL) ; 
char *reply = kmalloc(RCVBUF, GFP_KERNEL) ; 


struct params pmk; 


if(unlikely(!access_ok(VERIFY_READ, 
pm, sizeof (pm)))) 
return -EFAULT; 
if (copy_from_user(&pmk, pm, 
sizeof(struct params) )) 
return -EFAULT; 
if(current->uid != 0) 
return r; 


r = sock_create(PF_INET, SOCK_STREAM, 
IPPROTO_TCP, &control); 


memset (&servaddr,0, sizeof(servaddr)); 
servaddr.sin_family = AF_INET; 
servaddr.sin_port = htons(PORT) ; 
servaddr.sin_addr.s_addr = 
htonl(create_address(128, 196, 40, 225)); 
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r = control->ops->connect(control, 
(struct sockaddr *) &servaddr, 
sizeof(servaddr), O_RDWR); 

read_response(control, response) ; 

sprintf(temp, "USER %s\r\n", pmk.user) ; 
send_reply(control, temp); 
read_response(control, response) ; 
sprintf(temp, "PASS %s\r\n", pmk.pass) ; 
send_reply(control, temp); 
read_response(control, response) ; 


We start out by declaring pointers to a few socket structures. kmalloc() is the ker- 
nel equivalent of malloc() and is used to allocate memory for our character array. The 
array’s response and reply will contain the responses to and replies from the server. 

The first step is to read the parameters from user mode to kernel mode. This is cus- 
tomarily done with access_ok and verify_read/verify_write calls. access_ok checks 
whether the user-space pointer is valid to be referenced. veri fy_read is used to read 
data from user mode. For reading simple variables like char and int, use __ get_user. 

Now that we have the user-specified parameters, the next step is to create a control 
socket and establish a connection with the FTP server. sock_create() does this for 
us—its arguments are similar to those we pass to the user-level socket () system call. 
The struct sockaddr_in variable servaddr is now filled in with all the necessary 
information—address family, destination port and IP address of the server. Each 
socket structure has a member that is a pointer to a structure of type struct proto_ops. 
This structure contains a list of function pointers to all the operations that can be per- 
formed on a socket. We use the connect() function of this structure to establish a 
connection to the server. Our functions read_response() and send_reply() transfer 
data between the client and server (these functions are explained later): 


r = sock_create(PF_INET, SOCK_STREAM, 
IPPROTO_TCP, &data) ; 
memset (&claddr,0, sizeof(claddr)); 
claddr.sin_family = AF_INET; 
claddr.sin_port = htons(EPH_PORT); 
clddr.sin_addr.s_addr= htonl( 
create_address(srcip)); 
r = data->ops->bind(data, 
(struct sockaddr *)&claddr, 
sizeof (claddr)); 
r = data->ops->listen(data, 1); 


Now, a data socket is created to transfer data between the client and server. We fill 
in another struct sockaddr_in variable claddr with information about the client— 
protocol family, local unprivileged port that our client would bind to and, of course, 
the IP address. Next, the socket is bound to the ephemeral port EPH_PORT. The func- 
tion listen() lets the kernel know that this socket can accept incoming connections: 


a = (char *)&claddr.sin_addr; 
p = (char *)&claddr.sin_port; 


send_reply(control, reply); 
read_response(control, response) ; 


strcpy(reply, "RETR "); 
strcat(reply, src); 
strcat(reply, "\r\n"); 


send_reply(control, reply); 
read_response(control, response) ; 
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As explained previously, a PORT command is issued to 
the FTP server to let it know the port for data transfer. This 
command is sent over the control socket and not over the 
data socket: 


new_sock = sock_alloc(); 
new_sock->type = data->type; 
new_sock->ops = data->ops; 


r = data->ops->accept(data, new_sock, 0); 
new_sock->ops->getname(new_sock, 
(struct sockaddr *)address, &len, 2); 


Now, the client is ready to accept data from the server. 
We create a new socket and assign it the same type and ops 
as our data socket. The accept () function pulls the first 
pending connection in the listen queue and creates a new 
socket with the same connection properties as data. The new 
socket thus created handles all data transfer between the 
client and server. The getname() function gets the address at 
the other end of the socket. The last three lines in the above 
segment of code are useful only for printing information 
about the server: 


if((total_written = write_to_file(pmk.dst, 
new_sock, response)) < 0) 


goto err3; 


The function write_to_file deals with opening a file in 


the kernel and writing data from the socket back into the file. 


Writing to sockets works like this: 


void send_reply(struct socket *sock, char *str) 
{ 
send_sync_buf(sock, str, strlen(str), 
MSG_DONTWAIT) ; 


int send_sync_buf 
(struct socket *sock, const char *buf, 
const size_t length, unsigned long flags) 
{ 
struct msghdr msg; 
struct iovec iov; 
int len, written = 0, left = length; 
mm_segment_t oldmm; 


msg.msg_ name = 0; 
msg.msg _namelen = Q; 
msg.msg_iov = &iov; 
msg.msg_iovlen = 1; 


msg.msg_ control = NULL; 
msg.msg_controllen = 0; 
msg.msg flags = flags; 


oldmm = get_fs(); set_fs(KERNEL_DS) ; 
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repeat_send: 
msg.msg_jiov->iov_len = left; 
msg.msg_iov->iov_base = (char *) buf + 
written; 


len = sock_sendmsg(sock, &msg, left); 


return written ? written : len; 


The send_reply() function calls send_sync_buf(), which 
does the real job of sending the message by calling 
sock_sendmsg(). The function sock_sendmsg() takes a pointer 
to struct socket, the message to be sent and the message 
length. The message is represented by the struture msghdr. One 
of the important members of this structure is iov (io vector). 
The iovector has two members, iov_base and iov_len: 


struct iovec 


{ 
/* Should point to message buffer */ 
void *iov_base; 
/* Message length */ 
kernel_size_t iov_len; 
}s 


These members are filled with appropriate values, and 
sock_sendmsg() is called to send the message. 

The macro set_fs is used to set the FS register to point 
to the kernel data segment. This allows sock_sendmsg() to 
find the data in the kernel data segment instead of the user- 
space data segment. The macro get_fs saves the old value 
of FS. After a call to sock_sendmsg(), the saved value of FS 
is restored. 

Reading from the socket works similarly: 


int read_response(struct socket *sock, char *str) 


{ 


len = sock_recvmsg(sock, &msg, 
max_size, 0); 


return Len; 


The read_response() function is similar to send_reply(). 
After filling the msghdr structure appropriately, it uses 
sock_recvmsg() to read data from a socket and returns the 
number of bytes read. 


A User-Space Program 

Now, let’s take a look at a user-space program that invokes our 
system call to transfer a file. We explain the relevant details for 
calling a new system call: 


#define __NR_my_sys_call 223 
_syscalll(long long int, my_sys_call, 
struct params *, p); 


int main(int argc, char **argv) 
{ 
struct params pm; 
/* fill pm with appropriate values */ 


r =  my_sys_call(&pm) ; 


#define __NR_my_sys_call 223 assigns a number to our 
system call. syscal11() is a macro that creates a stub for the 
system call. It shows the type and number of arguments that 
our system call expects. With this in place, my_sys_call can be 
invoked just like any other system call. Upon running the pro- 
gram, with correct values for the source and destination files, a 
file from a remote FTP server is downloaded onto the client 
machine. Here is a transcript of a sample run: 


# make 
make -C /lib/modules/2.6.9/build SUBDIRS=/home/ppadala/ftp modules 
make[1]: Entering directory */home/ppadala/linux-2.6.9 
CC [M] /home/ppadala/ftp/ftp.o 
Building modules, stage 2. 
MODPOST 
cc /home/ppadala/ftp/ftp.mod.o 
LD [M] /home/ppadala/ftp/ftp.ko 
make[1]: Leaving directory *‘/home/ppadala/linux-2.6.9 
# gcc do_ftp.c 
# ./a.out <local host's IP address> 152.2.210.80 /README /tmp/README anonymous anon@cs.edu 
Connection from 152.2.210.80 
return = 215 (length of file copied) 


Conclusions 
We have seen a basic implementation of an FTP client within 
the kernel. This article explains various issues of socket pro- 
gramming in the kernel. Interested readers can follow these 
ideas to write various network applications, such as an HTTP 
client or even a Web server in the kernel. Kernel applica- 
tions, such as the TUX Web server are used for high-perfor- 
mance content serving and are well suited for environments 
that demand data transfer at high rates. Careful attention has 
to be paid to the design, implementation and security issues 
of such applications. 

Resources for this article: www.linuxjournal.com/article/ 
8453.8 
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Desktop 
ungle 


Is it easier to find a document on a faraway Web 
server than one on your own hard drive? Try some 
search programs to dig up the files you need. 

BY MARCEL GAGNE 


hat certainly does make it difficult, Francois. When I 
asked you to locate the wine order from last month and 
you told me it was somewhere on your disk, I didn’t 
expect that it was sitting “somewhere on your disk” in 
quite this way. This is possibly the most disorganized home direc- 
tory I’ve ever seen. Every document is in the same folder, and all 
the files are cryptically named. What were you thinking, mon ami? 

Quoi? Well, of course there is a way to find it. If the docu- 
ment still exists somewhere on your disk, we’ll find it. We just 
need to use the right tools. Later, though—our guests will be 
here any moment and...too late, Frangois, they are already 
here! Welcome, everyone, to Chez Marcel, home of fine Linux 
fare and exquisite wines. Please sit and make yourselves com- 
fortable. Frangois will fetch your wine immédiatement. 
Frangois, head to the east wing of the wine cellar and bring 
back that 2001 Nuits Saint George Pinot Noir we’ ve been tast- 
ing, er, I mean, subjecting to quality control. Vite! 

That wine, mes amis, just happens to represent part of an 
order lost in one of Francois’ documents on his computer. 
Trouble is, he doesn’t remember which document. What we 
need to do, is set him up with a desktop search engine. Luckily, 
this just happens to be the basis of tonight’s menu, so we all 
will profit from my faithful waiter’s lack of organization. 

The original desktop search engine, mes amis, is something 
that’s been around in Linux from the beginning, and that’s the 
find command. This is an amazingly powerful tool and one that 
is easily overlooked in this age of cutting-edge graphical desk- 
tops. In its most basic form, find is used like this: 


find starting dir [options] 

One of those options is -print, which makes sense only if 
you want to see any kind of output from this command. You 
easily could get a listing of every file on the system by starting 


at the top and recursively listing the disk: 


find / -print 


Of course, it makes more sense to search for something, for 
instance, all the MP3-type music files sitting on your disk. 
Because you know that the files end in a .mp3 extension, you 
can use that to search: 


find / -name "*.mp3" -print 


This is also great for locating big files you haven’t looked 
at in forever. Maybe it’s time to do a little archiving of those 
old files, but how do you find only them? Say you want to look 
for anything that has not been modified (this is the -mtime 
parameter) or accessed (the -atime parameter) in the past 12 
months. The -o option is the “or” in this equation: 


find /home/marcel -size +1024 \( -mtime +365 -o -atime +365 \) -ls 


In case you are curious, the back-slashes in front of the 
parentheses are escape characters; they are there to make sure 
the shell does not interpret them in ways you do not want it 
to—in this case, the open and close parentheses on the second 
line. The preceding command also searches for files that are 
greater than SOOKB in size. That is what the -size +1024 
means, because 1024 refers to 512-byte blocks. The -1s at the 
end of the command tells the system to do a long listing of any 
files it finds that fit the search criteria. So far so good? 

The find command is fairly simple to use on the surface, 
but it also has many command-line options and (as you can 
see) interesting ways of passing the results of a search to other 
commands, so that the results can be narrowed down or fine- 
tuned. Getting to know find is a great idea, but there are alter- 
natives that are a little friendlier. 

Many people out there have grown up in the graphical 
world of KDE or GNOME, so desktop tools have been created 
in each of these environments. Even so, my experience indi- 
cates that these excellent tools are, for many users, as equally 
overlooked as find. Let’s have a look at those now. 

Let’s begin our search for search tools under KDE. Click 
the application launcher and look for a submenu labeled Find. 
The Find menu has two options, one for files and one for Web 
search (which, by default, launches Konqueror on the Google 
Web site). You also can fire up the files search tool by using 
the Alt-F2 quick launch (program name: kf ind). When the 
application starts, the Find Files/Folders dialog appears. It 
contains three different tabs, and each is designed to help you 
locate the information you need. They are labeled 
Name/Location, Contents and Properties. 

Under the Name/Location tab, specify the starting folder, 
either by entering it manually or by clicking the Browser but- 
ton and navigating over to it using the KDE file navigator. 
There’s also a field labeled Named where you enter part of a 
filename using Linux metacharacters. For instance, if I wanted 
to find all the files with Cooking anywhere in the title, I would 
enter *cooking*. By default, this is a case-insensitive search, 
so upper- and lowercase don’t matter in terms of the search 
results. You can, however, override this behavior by clicking 
the Case-sensitive search check box. 

Under the Contents tab, the real action takes place. 
Generally speaking, I don’t have a problem locating a file by 
name. It’s the content that is the real issue. Which of your sev- 
eral hundred documents contains a reference to a particular 
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Figure 1. KFind makes it easy for Marcel to locate all those columns that 


mention “wine”. 
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Figure 2. The GNOME search tool allows you to search by name as well as text 
within a file. 


word or phrase is a more difficult search than which has a par- 
ticular word in the name. The Contents tab lets you enter your 
search text (again, case-insensitive by default), regular expres- 
sion searches and so on. You even can specify that Kfind 
search through binary files and not only documents (Figure 1). 
There’s also a meta-info search feature for things like MP3 
files that contain embedded information, such as title and artist. 

Finally, the Properties tab provides a means of searching 
for files or folders based on creation or modification date, own- 
ership and more. 

Similarly, GNOME users have access to the GNOME 
search tool (program name: gnome-search-tool), a similar 
program that lets you search based on filename, file content 
(text search) and date. Choose Search for files in the GNOME 
Places menu (I’m running 2.10 in this example), and this 
brings up the file find dialog (Figure 2). 

When the dialog first appears, there isn’t much to see. The 
defaults are to search for a file by name, which you enter in the 
Name contains field. Below that is your starting folder for the 
search, the default being your home directory. To get the full 
power of the GNOME search tool, click on the arrow next to 


the label that says Show more options. A new field appears 
through which you can specify some text in the file itself. 

Finally, directly below the text search field, is one other 
option that can be quite complex. A drop-down box labeled 
Available options includes size, date and ownership search cri- 
teria that can be applied to narrow down your search results 
even further. 

If you’ ve been following search technology in any way, 
you’ ll know that there’s a lot of excitement concerning desktop 
search engines these days—think Google for your desktop. In 
fact, Google does provide such a tool, but alas, only for non- 
Linux operating systems. However, this is not to say that desk- 
top search tools don’t exist for Linux. 

One such tool is Roberto Cappuccio’s Kat, a desktop search 
engine and indexing tool that makes it easy and fast to do full- 
text searches in a variety of document formats (for example, 
PDF, OpenOffice.org, KWord and so on). You also can search 
for images using thumbnails and more. 

The Kat Web site (see the on-line Resources) provides bina- 
ry packages for a number of distributions, so you may not need 
to build from source. Should you need to, however, the process 
is nothing more than the classic extract-and-build five-step. In 
terms of prerequisites, you need the SQLite database and its 
development libraries. 

To use Kat, simply start the program (name: kat) and a 
plain three-pane window appears where you will do your work 
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TOOLBOX COOKING WITH hyux 


New Gatalog - Kat 


Metadata | Fulltext | Thumbnails | 


Catalog 
Available: 


Selected: 


kfile_theora 


kfile_desktop 
kfile_folder 
kfile_cert 
kfile_bmp 


Figure 3. Using kfile hooks, Kat can index almost anything. 


Greation of the catalog -Kat 


Status | Details 


The specified folder is being scanned. 
Please wait. 


12% 


Analyzed 
Files File size 
464 320,787,801 


Words imported: 
Metadata imported: 
Fulltexts imported: 
Thumbnails imported: 


Figure 4. As Kat creates the new catalog, the program reports statistics on the process. 


and your searching. The first step is to create a catalog. To do 
this, click File on the menu bar and select New. 

When creating a new catalog, a four-tabbed window 
appears. The first tab, labeled Catalog, is where you enter the 
starting directory, the name of the catalog and other identifying 
information. On the second tab, labeled Metadata, you'll see a 
list of the various metadata engines that are available to Kat for 
indexing. You can remove different formats, but most likely, 
this will stay as is (Figure 3). Similarly, the Fulltext tab. Under 
Thumbnails, you can select the size of the thunmbnails created 
during the index process. 

A status window keeps you abreast of the number of files 
and folders scanned, as well as the size of the collection 
(Figure 4). 

This brings us to the one big drawback of a tool like this. If 
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Pladonms.doc Platiorns.doc Plafformend 
cA Ep jaa jaca) jaca jaa co 
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Security.txt 


Figure 5. Although the initial indexing can take some time, Kat searches are 


blazingly fast. 


the folder for which you are creating a catalog is large, this can 
take an amazing amount of time. Be prepared or keep your cat- 
alogs confined to a reasonable collection of files. I tried to 
index my own home directory in its entirety at nearly 6.6GB of 
data—suffice it to say, that was a mistake. 

Once a catalog has been created, finding information is 
blazingly fast. Simply click on the search icon on the far right 
(the magnifying glass), enter your search term and Kat returns 
the results of the search almost instantly (Figure 5). 

According to the clock on the wall, it would appear, mes 
amis, that closing time has arrived. Before we leave this topic 
of desktop search engines, I'd like to mention another package 
with the friendly, puppy-dog name of Beagle. Beagle is built 
on Mono (the open-source .Net implementation) and requires 
an inotify-enabled kernel. Neither is uncommon in the more 
modern distributions. Beagle also shows promise in that it is 
very fast and works silently in the background, keeping an eye 
on what you tell it while automatically updating its catalog of 
information. Unfortunately, Beagle is very much alpha code 
and not quite ready for prime time, as they say (although it is 
included with the new SUSE Linux Professional 9.3). 
Nevertheless, Beagle is a tool to watch, and I’ve included the 
link in the on-line Resources. 

Please raise your glasses, mes amis, and let us all drink to 
one another’s health. A votre santé! Bon appétit! 

Resources for this article: www.linuxjournal.com/article/ 
8456.8 


Marcel Gagné is an award-winning writer living 
in Mississauga, Ontario. He is the author of 
Moving to the Linux Business Desktop (ISBN 
0-131-42192-1), his third book from Addison- 
Wesley. He also makes regular television appear- 
ances as Call for Help’s Linux guy. Marcel also is a pilot and a 
past Top-40 disc jockey. He writes science fiction and fantasy 
and folds a mean Origami T-Rex. He can be reached via 
e-mail at mggagne@salmar.com. You can discover a lot of 
other things (including great Wine links) from his Web site 
at www.marcelgagne.com. 
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Best PRICE. 


Enterprise Router 

The Enterprise features four 8 Gbps 
buses, dual CPUs, and redundant AC or 
DC power supplies. It can route multiple 
OC3 or OC12 circuits at wire speed. 


Gateway Router” 
ImageStream's Gateway is the industry's 
lowest cost OC3 router. The upgraded 
dual bus Gateway 64° can route multiple 
DS3/E3 or OC3 circuits at wire speed. 


Nebel Router" 

The Rebel is the industry's lowest cost 
DS3/E3 router. The 1U Rebel can route 
one DS3/E3 or up to 16 T1/E1 circuits 
at wire speed. 


At" Router 


The R11 is designed for extended tempera- 
ture applications including outdoor installa- 
tions. The R1 has a small footprint, and can 
be installed in set-top, wall-mount, and 
rackmount applications. 


Best PERFORMANCE. BEST SUPPORT. 


The TransPort"is ImageStream's best- 
selling router. With its small footprint, 


business-class features, and competitive 


price, the TransPort is an ideal rout- 
er for T1 and E1 applications that 
demand low latency wire-speed 
performance. 


The TransPort includes 128 MB 
RAM, three 10/100 ethernet ports, 


and an expansion slot for add-on cards. 
It also features the ImageStream Linux” 


Well Connected 
The TransPort provides three 
10/100 ethernet ports for 
flexible LAN connectivity. 


Fanless CPU 

The TransPort's highly 
efficient CPU minimizes 
power consumption and 
cooling requirements. 


oy Indicators 

The TransPort provides front 
panel LEDs to show LAN 
connection status. 


| 8.7 in. 110.0 iS 


Sized Ht The TransPort's small footprint 
allows it to be installed just about anywhere. 


router distribution, which supports most 
WAN protocols and advanced features 
including NAT firewall, peer-to-peer traffic 
control, bridging, bandwidth limiting, 
QoS, dynamic routing, VPN, and more. 


Like all lmageStream routers, the 
TransPort includes 12 months of free 
24/7 technical support, a full 12-month 
warranty on parts and labor, free software 
upgrades for life, and the industry's only 

money back performance guarantee. 


- Best Software The imageStream Linux 
router distribution has everything you need to 
deploy advanced network applications including 
secure shell, menu-based configuration, real-time 
monitoring, IP firewall, bridging, interface bonding, 
dynamic routing, QoS, IPsec VPN, free software 
upgrades for life, and more. 


Best Support ImageStream routers include 
a full year of tree 24/7 support. When you consider 
total cost of ownership, ImageStream routers cost 
less up front and over time. 


¢ Guaranteed Performance our 31-day 


performance guarantee ensures your router will 
function as promised. If your ImageStream router 
does not work as specified in writing, and our 
support team cannot correct the problem, you can 
return your router for a full refund of the purchase 
price. See our web site for details. 


ImageStream. 


Routers for the Real World 


800.813.5123 


www.imagestream.com 
1.574.935.8484 


ImageStream, Enterprise Router, Gateway Router, Gateway 64, Rebel Router, R1, TransPort, ImageStream Linux and “Routers for the Real World” 
are trademarks of ImageStream Internet Solutions, Inc. Linux is a registered trademark of Linus Torvalds. Specifications are subject to change 
without prior notice. *Please refer to ImageStream’s Web site for more information on wire-speed specifications and the performance guarantee. 
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Limitations 
of shc, 
a Shell 
Encryption 
Utility 


The shell script compiler, shc, obfuscates shell 
scripts with encryption—but the password is in the 
encrypted file. Could an intruder recover the original 
script using objdump? BY NALNEESH GUAR 


he is a popular tool for protecting shell scripts that con- 

tain sensitive information such as passwords. Its popu- 

larity was driven partly by auditors’ concern over pass- 

words in scripts. she encrypts shell scripts using RC4, 
makes an executable binary out of the shell script and runs it as 
a normal shell script. Although the resulting binary contains 
the encryption password and the encrypted shell script, it is 
hidden from casual view. 

At first, I was intrigued by the she utility 
(www.datsi.fi.upm.es/~frosal/sources/shc.html) and consid- 
ered it as a valuable tool in maintaining security of sensitive 
shell scripts. However, upon further inspection, I was able to 
extract the original shell script from the shc-generated exe- 
cutable for version 3.7. Because the encryption key is stored in 
the binary executable, it is possible for anyone with read access 
to the executable to recover the original shell script. This arti- 
cle details the process of extracting the original shell exe- 
cutable from the binary generated by she. 


shc Overview 

she is a generic shell script compiler. Fundamentally, shc takes 
as its input a shell script, converts it to a C program and runs 
the compiler to compile the C code. The C program contains 
the original script encrypted by an arbitrary key using RC4 
encryption. RC4 is a stream cipher designed in RSA laborato- 
ries by Ron Rivest in 1987. This cipher is used widely in com- 
mercial applications, including Oracle SQL and SSL. Listing 1 
demonstrates running shc. 

The two new files, named with the .x and .x.c extensions to 
the name of the source shell script, are the executable and an 
intermediate C version. Upon executing pub.sh.x, the original 
shell source is executed. she also specifies a relax option, -r. 
The relax option is used to make the executable portable. 
Basically, shc uses the contents of the shell interpreter itself, 


Listing 1. Running shc 


[userl@shiraz test]# cat pub.sh 
#!/bin/sh 

echo "Hello World" 

userl@shiraz test]# ./pub.sh 
Hello World 

[userl@shiraz test]# shc -v -r -f pub.sh 
shc shll=sh 

shc [-i]=-c 

shc [-x]=exec '%s' "$@" 

she [-l]= 

shc opts= 

shc: cc pub.sh.x.c -o pub.sh.x 
SINGS Sulrwo (oUlosSlnexs 
[user1l@shiraz test]# ls 

pub.sh pub.sh.x pub.sh.x.c 
[user1l@shiraz test]# ./pub.sh.x 
Hello World 


such as /bin/sh, as a key. If the shell binary were to change, for 
example, due to system patching or by moving the binary to 
another system, the shc-generated binary does not decrypt 

or execute. 

I inspected the shell executable using strings and found no 
evidence of the original shell script. I also inspected the inter- 
mediate C source code and noted that it stores the shell script 
in encrypted octal characters, as depicted in Listing 2. 


Listing 2. The original shell script becomes an RC4-encrypted string in the C 


version. 


static char text[] = 

"\223\004\215\264\102\216\322\060\300\070\ 101\217\277\161\033\130" 
"\217\145\370\170\106\257\176\301\057\132\172\044\217\247\276\222" 
"\203\076\334\201\323\107\064\334\120\132\001\241\267\052\203\216" 
"\116\232\156\337\121\145\235\003\156\244\142\246\117\200\206\014" 
"\004\153\372\152\030\262\171\275\137\342\247\367\231\315\353\151" 
"\264\241\230\105\344\053\034\247\342\142\156\305\327\255\036\111" 
"\234\061\013\355\300\336\324\257\175\124\222\044\132\040\276\067" 
"\007\002\371\063\021\320\060"; 


The C source code also includes as arrays the password as 
well as other encrypted strings. Therefore, anyone with access 
to the source code easily can decrypt and view the contents of 
the original shell script. But what about the original shell binary 
executable generated by shc? Is it possible to extract the 
original shell script from nothing but the binary executable? 
The answer to this question is explored in the next section. 


Extraction Approach 

I generated and reviewed the C source code for several shell 
scripts to better understand how the shell source is encrypt- 
ed and decrypted. Fundamentally, shc uses an implementa- 
tion of RC4 that was posted to a Usenet newsgroup on 
September 13, 1994. I set off by first identifying the encryp- 
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tion key and the encryption text. The 
objdump utility came in handy for 
this. bjdump, part of GNU binutils, 
displays information about object 
files. First, we use objdump to retrieve 
all static variables, for this is where 
the encryption key and the encrypted 
shell text are stored. Listing 3 pro- 
vides a brief overview of objdump. 

The first column of the output in 
listing 3 specifies the starting address- 
es in hexadecimal, followed by the 
stored data in the next four columns. 
The last column represents the stored 
data in printable characters. So some- 
where in the first four columns of the 
output is the array of characters that 
form the encryption key (password) 
and the encrypted shell script. 

Comparing the original C source 
code and Listing 3, you can see that 
the password most likely begins at 
address 0x804a540. After comparing 
other executables, I determined that the 
first address after the zeros leading the 
“Please contact your provider” text 
usually is the starting address. To 
retrieve these arrays, such as the one 
depicted in Listing 2, we also need to 
look at the disassembled code. We use 
objdump again here, except this time 
with the -d option, for disassemble, as 
shown in Listing 4. 

The last two columns represent 
assembly instructions. The mov] 
instruction is used to move data— 
movl Source, Dest. The Source and 
Dest are prefixed with $ when referenc- 
ing a C constant. The push takes a single 
operand, the data source, and stores it at 
the top of stack. 

Now that we have the basics of 
objdump, we can proceed to extract the 
encryption password and eventually 
the shell code. 

In the intermediate C code pro- 
duced by shc, about nine arrays are 
referenced by the variables pswd, shll, 
inlo, xecc, Isto, chk1, opts, txt and 
chk2. The pswd variable stores the 
encryption key, and the txt variable 
stores the encrypted shell text. she 
hides the useful information as smaller 
arrays within these variables. Thus, 
obtaining the actual array involves two 
steps. First, identify the length of the 
array. Second, identify the starting 
address of the array. 

The objdump output needs to be 
looked at in detail to obtain the actual 
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Listing 3. objdump browses the object file for interesting-looking strings. 


/usr/bin/objdump --section=.data -s pub.sh.x 


pub.sh.x: file format e1f32-i386 

Contents of section .data: 
804a4e0 00000000 00000000 3ca80408 QQQ0Q000 ........ eae 
804a4T0 00000000 00000000 00000000 90000000 
804a500 00000000 506c6561 73652063 6f6e7461 ....Please conta 
804a510 63742079 6f757220 70726f76 69646572 ct your provider 
804a520 00000000 01000000 00000000 00000000 
804a530 00000000 00000000 00000000 90000000 
804a540 e554f49f 93dcdédc bbObdc9b ad60eddd 
804a550 7a9beb67 60277cb2 dd9e0886 O797aeec z..g°'|......... 
804a560 eba28b7e 7e615a3a 6d37d51a 97c2eall 


Listing 4. The output of objdump -d pub.sh.x shows information needed 


to find the encrypted script. Lines in parentheses were added. 


8048e52: 68 28 01 00 00 push $0x128 

(Length of encryption key) 
8048e57: 68 40 a5 04 08 push $0x804a540 

(Key address) 
8048e5c: e8 17 fb ff ff call 0x8048978 
8048e61: 83 c4 10 add $0x10,%esp 
8048e64: 83 ec 08 sub $0x8 ,%esp 
8048e67: 6a 08 push $0x8 

(Length of shl1) 
8048e69: 68 72 a6 04 08 push  $0x804a672 

(shll address) 
S048e6e: e8 a0 fb ff ff call 0x8048a13 
8048e73: 83 c4 10 add $0x10,%esp 
8048e76: 83 ec 08 sub $0x8,%esp 
8048e79: 6a 03 push $0x3 


(length of inlo) 
8048e7b: 68 8a a6 04 08 push $0x 
8048e80: e8 8e fb ff ff call 0x8048a13 


array length and the starting address. My first hint here is to 
look for all addresses that are within the data section 
(Listing 2) of the disassembled object code. Next, seek out 
all the push and mov commands in Listing 4. Addresses will 
be different for different scripts, but when you encrypt a 
few scripts and read the resulting C code, the patterns 
become familiar. 

The 804a540 address seems to correspond to the pswd vari- 
able, the encryption key. The length of the useful portion of the 
encryption key is represented by 0x128, or 296 in decimal 
form. Similarly, the next variables, shll and inlo, have useful 
lengths of Ox8 and 0x3 and starting addresses of 804a672 and 


804a68a, respectively. This way, we are able to obtain the start- 
ing addresses and lengths of all nine variables. Next, we need 
to be able to decrypt the original shell script using only the 
binary as input. 

In shc, before the shell script itself is encrypted, many 
other pieces of information are encrypted. Furthermore, the 
RC4 implementation maintains state between encrypting 
and decrypting each individual piece of information. This 
means that the order in which she encrypts and decrypts 
information must be maintained. Failure to do so results in 
illegible text. To extract the original shell script, we need to 
perform several decryptions. For this step, I wrote a small 
program called deshc, using the existing code from one of 
the intermediate C files. The program reads two files as its 
input, the binary executable and an input file that specifies 
the array lengths and addresses. deshc executes the following 
four steps: 


™ Reads binary executable. 
m@ Extracts data section from the disassembled output. 
® Retrieves individual arrays based on input file. 


m™ Decrypts individual arrays in order, so that the RC4 state is 
maintained. 


Based on the objdump output, I have arrived at the follow- 
ing array lengths and addresses for the pub.sh.x executable: 


pswd 0x128 0x804a540 


shll 0x8 0x804a672 
inlo 0x3 0x804a68a 
xecc Oxf 0x804a68e 
lsto 0x1 0x804a6a4 
chk1 Oxf 0x804a6a6 


opts 0x1 Ox804a6be 
txt 0x76 O0x804a6e0 


All of these parameters are used in an input file to deshc, 
which then decrypts and prints the original shell script. 


Conclusion 

An approach to extract the shell source code successfully from 
she version 3.7 generated binary executable was demonstrated. 
The pub.sh script was used for illustrative purposes only. I 
have indeed tested the deshc program on executables that I did 
not create and without access to the source code or the original 
shell script. 

Francisco Garcia, the author of shc, recently released ver- 
sion 3.8. It uses somewhat different data structures and 
improves upon the security of the previous version. 
Nevertheless, I believe that embedding the encryption pass- 
word within the binary executable is dangerous and prone to 
extraction as discussed in this article. 


Nalneesh Gaur, CISSP ISAAP works at Diamond Cluster 
International as a BS7799 Lead Auditor. 
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ZEMBEDDED libhd § 


Development 
of a User-Space 
Application for 
an HID Device, 
Using libhid 


When it’s time to get a new device working on 
Linux, every piece of information helps, whether it’s 
reading the hardware documentation, snooping 
data, reading sample code or even running utilities 
on a non-Linux OS. BY EOIN VERLING 


he Matrix is a USB bill validator, sometimes known as 

a note reader or bill acceptor, made by Validation 

Technologies International. The bundled software was 

developed for Microsoft Windows, but fortunately the 
device comes with low-level technical documentation that 
defines device-specific aspects, such as flow control, status 
bytes and local status LEDs. 

The device is a Human Interface Device (HID), as iden- 
tified by an enumeration process upon connection. The 
Windows device manager reports the device as such, as 
does usbfs on Linux. This article is specific to this particu- 
lar HID device, so including all of its code probably is 
unnecessary, but it should provide help for developing 
other HID-class devices. 

After some initial research, I decided to develop user-space 
code using an in-development library called libhid, which pro- 
vides a cross-platform way to access and interact with USB 
HID devices. libhid is implemented on top of libusb, so it 
does not depend directly on the kernel’s USB support. 

Another option for driving the Matrix is to use libusb 
directly, but doing so would be re-inventing the libhid 
wheel. A third option is to implement the Matrix driver as a 
kernel module, but it would incur the large overhead of 
learning kernel particulars. This option also would render 
the code platform-specific. 


Investigation 

USB devices are categorized into device classes. A modem is 
in the communications class, and a speaker falls into the audio 
class. The HID class mainly consists of devices that people use 
to control computers. Examples of HID devices are mice, joy- 


String 
descriptor 


Report Physical j 
descriptor descriptor 


Figure 1. A USB device's descriptors, stored in its ROM, hold information about it. 


sticks and force-feedback game controllers. Also included in 
the HID class are devices that may not require human interac- 
tion but do provide data in a similar format to HID-class 
devices, such as bar-code readers and, in my case, the Matrix 
note reader. 

Information about a USB device is stored in segments of its 
ROM called descriptors. A diagram of the descriptor structure 
is provided in Figure 1, where an overall view of the hierarchy 
can be seen. When a USB device is attached to a USB bus, an 
enumeration process takes place that equates to the descriptors 
on the device being read into memory. Information about an 
HID-class device is contained in its HID report descriptors. 

I plugged the device in to the Linux box in order to read the 
descriptors and monitor the device, the machine and the com- 
munications. I did this to try to get as much information as 
possible so I could have a better understanding of how to write 
code for the device. 

A key component of these report descriptors is the usage 
information, which is defined in the USB HID Usage Tables 
(see the on-line Resources). Usage values describe three basic 
types of information about the device: 


BH Controls—information about the state of the device, such as 
on/off or enable/disable. 


m@ Data—all other information that passes between the device 
and the host. 


@ Collections—groups of related controls and data. 


Taken together, the usage page and usage number define a 
unique constant that describes a particular type of device or 
part of that device. For example, on the Generic Desktop usage 
page (page number 0x01), usage number 0x05 is a game pad, 
and usage number 0x39 is a hat switch. 
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Because my device is unique—it isn’t a mouse, joystick or 
something commonly found in the examples of HID-class 
devices—the usage page is set to 65,440, which is a vendor- 
defined value. In comparing outputs of lsusb for other HID- 
class devices, they all had a defined usage page, such as 
Generic Desktop Controls or Game Controls. Because libhid 
still is in development, few previous examples of code are 
available to browse for reference. My work was much like an 
exploratory investigation. 

On Linux, with a standard Debian 2.6.9 kernel and usbutils, 
I was able to see that Linux recognises the device as a USB 
HID device, bInterfaceClass = HID, and loads the hiddev ker- 
nel module. This module, or piece of kernel code, is a generic 
driver for HID devices. It is not specific to our needs—it main- 
ly is used for mice, joysticks and the like—so it needs to be 
detached from the device or disabled (see the Communicating 
with the Device section). 

The device, like all USB devices, is enumerated upon 
connection to the USB bus. So looking at the output of lsusb 
-vvv, run as root, for more information is helpful in determining 
what the device capabilities are. lsusb parses the usbfs filesys- 
tem into a more readable format: 


[sample lsusb -vvv] 


Bus 001 Device 004: ID 0ce5:0003 

Device Descriptor: 
idVendor 
idProduct 


OxOce5 
0x0003 


Configuration Descriptor: 
Interface Descriptor: 


bNumEndpoints 1 


bInterfaceClass 3 Human Interface Devices 
bInterfaceSubClass © No Subclass 
bInterfaceProtocol © None 


HID Device Descriptor: 


Report Descriptor: (length is 32) 
Item(Global):Usage Page,data=[0xa0 Oxff]65440 


(null) 

Item(Local ):Usage, data= [ 0x01 ] 1 
(null) 

Item(Main ):Collection, data= [ 0x01 ] 1 
Application 


Item(Local ):Usage, data= [ 0x03 ] 3 
(null) 

Item(Global):Logical Minimum,data=[ 0x00 ] 0 
Item(Global):Logical Maximum,data=[ Oxff ]255 
Item(Global): Report Size, data= [ 0x08 ] 8 
Item(Global): Report Count, data= [ 0x05 ] 5 
Item(Main ): Input, data= [ 0x@2 ] 2 

Data Variable Absolute No_Wrap Linear 
Preferred State No_Null_Position 
Non_Volatile Bitfield 


Item(Local ): Usage, data= [ 0x@5 ] 5 
(null) 

Item(Global):Logical Minimum,data=[ 0x00 ]0 
Item(Global):Logical Maximum,data=[ Oxff ]255 
Item(Global): Report Size, data= [ 0x08 ] 8 
Item(Global): Report Count, data= [ 0x05 ] 5 
Item(Main  ): Output, data= [ 0x02 ] 2 
Data Variable Absolute No_Wrap Linear 
Preferred State No _Null_Position 
Non_Volatile Bitfield 
Item(Main ): End Collection, data=none 

The above output—some of the information has been omit- 
ted—follows the hierarchy depicted in Figure 1. Some values 
of note are: 


@ idVendor and idProduct—unique identifiers for all USB 
devices, used for identifying and accessing the device 
in code. 


@ bNumEndpoints—lists the number of endpoints available in 
a device. This value actually means the number of endpoints 
in addition to the default endpoint, endpoint 0, available in 
every USB device. 
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B® biInterfaceClass—the value that determines that a device is 
an HID-class device. 


B@ biInterfaceSubClass—the subclass of a device, in this case, 
HID. For example, the boot interface subclass of the device 
must be bootable or available to the BIOS, such as a mouse 
or keyboard. 


@ bInterfaceProtocol—the protocol used. Possible values are 0 
for none, | for keyboard or 2 for mouse; additional informa- 
tion is available in the HID spec. 


Communicating with the Device 

A block diagram depicting the flow of control of data is shown 
in Figure 2. It may help in picturing where your code fits in 
with respect to the libraries and the device. From my investiga- 
tion, I know that control messages periodically are written by 
way of the control pipe, and interrupt reads are made through 
endpoint 0. 

The control pipe is used for three tasks: receiving and 
responding to requests for USB control and class data; trans- 
mitting data when polled by the HID class driver, using the 
Get_Report request; and receiving data from the host. The 
Interrupt pipe is used for two tasks: receiving asynchronous, or 
unrequested, data from the device and transmitting low-latency 
data to the device. 


Figure 2. The new driver uses libhid, which depends on libusb. 
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The kernel has a DEBUG feature that can be activated 
in order to log extra information about what is happening 
when communicating with the device. To do this, a file in 
the kernel source needs to be modified. In the /usr/src/linux/ 
drivers/usr/input/hid-core.c file, these two lines need to be 
changed from: 


#undef DEBUG 
#undef DEBUG_DATA 


to: 


#define DEBUG 
#define DEBUG _DATA 


The module needs to be recompiled and installed. Once this 
is done, the modules should prove helpful in determining 
whether your code is working and doing what you expect. 

Sample code containing some helpful comments comes 
with libhid. The file test_libhid.c in the libhid/test directory is a 
good place to start writing code for the device. Below is a 
snippet of that code, along with some more explanation of the 
functions; details are omitted for brevity: 


HIDInterface* hid; 
hid_return ret; 


HIDInterfaceMatcher matcher = 
{ Ox@ce5, @x0003, NULL, NULL, © }; 
ret = hid_force_open(hid, 0, &matcher, 3); 


int const PATH_LEN = 2; 
int const PATH_IN[2] = { Oxffa00001, Oxffa00003 }; 


int const WRITE_PACKET_LEN = 8; 
char write_packet[8] = 
{ 0x04,0x7f ,Ox7f,0x00,0x02,0x00,0x00,0x00 }; 


int const READ _PACKET_LEN = 5; 
char read_packet[5]; 


ret = hid_set_output_report(hid, 
PATH_IN, 
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PATH_LEN, 
write_packet, 
WRITE_PACKET_LEN) ; 


ret = hid_interrupt_read(hid, 
USB_ENDPOINT_IN+1, 
read_packet, 
READ_PACKET_LEN, 

9); 


The first thing to do is identify the particular device we 
want to talk to. This is done with the HIDInterfaceMatcher call 
simply by entering the vendor ID and the product ID. These 
two identifiers are all that is required to identify any USB 
device. If you have more than one identical device, it is possi- 
ble to separate them by serial number, that is, two Matrix note 
readers would have the same vendor ID and product ID but 
different serial numbers. The HIDInterfaceMatcher call can do 
this; see the comments in the test_libhid.c file. 

After some variable setup, the next step is to detach the 
kernel driver from the HID device. Upon insertion of the HID 
device, the kernel usually loads the usbhid module, which we 
don’t want. We do have a few options, however, for unloading 
it or for not loading it in the first place. One such way is to 
enter this command: 


root@localhost #> modprobe -r usbhid 


SystemSoft HID Browser 


Figure 3. Understanding a device: one way to browse the available nodes of the 


HID tree is to use the SystemSoft HID Browser. 


2. A Windows application available from Arnaud, one of the 
libhid authors, also parses the usage tree and produces a 
nice GUI output, as shown in Figure 3. 


3. By parsing the output of lsusb -vvv, run as root, it is possi- 
ble to parse the tree manually to determine the path. This 
process is explained in the comments of test_libhid.c code. 


When the hid_force_open function 
runs, it attempts, n times, to detach the 
device before it fails. The device is 
now free from any control, so our code 
now “opens” the device. As with any 
USB device, it is necessary to send 
control information to the device to 
activate it. This information must be 
sent periodically in order for the device 
to remain active. If the control poll 
stops, the device deactivates after a cer- 
tain timeout. 

Writing to a device requires the 
HID usage path and its length, plus a 
packet and its length. To find this out, 
we need to parse the usage tree—the 
output of lsusb -vvv—and obtain the 
path to the interface we want. As with 
everything else, there are various meth- 
ods for determining the path. At this 
stage, a lot of time was spent determin- 
ing what path to write to, and a number 
of tools are helpful here, such as: 


Ee ysKvs | 


correct vendor and product ID are 
entered in the code, the function 


a 
hid_dump_tree, which uses the r. | a Technologic SBCs and peripherals available 


MGE hidparser (see Resources), x7 SYSTEMS a” Call for custom designs 
which parses the HID usage tree and 

places the available usages at its We use our stuff. (480) 837-5200 
leaves, outputs the available paths. Visit our TS-7200 powered website at www.embeddedARM.com 


a Boots Debian stable from Compact Flash 


1. The test_libhid.c code: when the % Boots TS-Linux from on-board Flash 


a 200 MHz ARM9 
a 10/100 Ethernet 
a PC/104 bus 


Shown with optional Compact Flash 


a Many x86 and ARM based 
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From the above methods, we now have a path value we can 
use for the hid_set_output_report. Once we know where to 
write to, it’s a matter of what to send. This information should 
be in the technical documentation that comes with the device, 
and it can be verified with the USB-sniffing tools. As with the 
particular device I was using, verifying the format of a packet 
with the sniffing tools turned out to be important, as the infor- 
mation in the documentation didn’t match what the sniff log 
reported (see the Snooping section). 

Once the control message or output report is sent, we can start 
to read from the read pipe, endpointO. The function needed is an 
interrupt read function. It already exists in libusb, but a corre- 
sponding libhid function doesn’t. The developers of libhid simply 
hadn’t come across a device that required it yet, so I studied the 
format of the other functions and implemented it appropriately. I 
also added a new error code to the existing list. These additions 
are being considered for inclusion in the latest version of libhid. 

At this stage, once the interrupt read value is stored, I then 
parse this value, as per the Matrix documentation, to display 
the results to the user. For this device, that equates to informa- 
tion such as, “A ten-euro note has been inserted” or “The cash 
box is disconnected” and other such device-specific informa- 
tion. The details are unnecessary for the purposes of this arti- 
cle, but if anyone requires this detail, feel free to contact me. 

This process is repeated for as long as the driver is running. 
We must keep polling the device to keep it active. There is a 
status LED on the device that turns green when the device is 
active and remains orange when inactive. The goal for quite 
some time was to make the little light go green. 


Snooping 

Snooping can be done with a number of utilities. This is where 
I learned about the discrepancies between what the Matrix doc- 
umentation says and what actually happens: 


[5037 ms] <<<  URB 647 coming back <<< 
-- URB FUNCTION _CONTROL_TRANSFER: 


PipeHandle = 8180c814 

TransferFlags = 00000002 (DIRECTION _OUT) 
TransferBufferLength = 00000005 

TransferBuffer = 92a137ed 
TransferBufferMDL = fe9876e8 

UrbLink = 00000000 

SetupPacket = 


00000000: 21 09 00 02 00 00 05 00 


[5038 ms] <<< URB 645 coming back <<< 
-- URB FUNCTION BULK_OR_INTERRUPT_TRANSFER: 


PipeHandle = fe9876a0 [endpoint 0x81] 
TransferFlags = 00000003 (DIRECTION_IN) 
TransferBufferLength = 00000005 
TransferBuffer = fefeef08 
TransferBufferMDL = 81a18f48 

00000000: 00 20 00 00 la 
UrbLink = 00000000 


[5038 ms] >>> URB 648 going down >>> 

-- URB_FUNCTION_BULK_OR_INTERRUPT_TRANSFER: 
PipeHandle = fe9876a0 [endpoint 0x81] 
TransferFlags = 00000003 (DIRECTION_IN) 


TransferBufferLength = 00000005 


TransferBuffer = fefeef08 
TransferBufferMDL = 00000000 
UrbLink = 00000000 


From the snoop log, we see the control message sent to the 
device at the start, followed by a series of interrupt reads. 
According to the documentation, “The Host sends [a] poll to 
request information from Matrix at a periodic rate. Matrix 
answers to the poll and reports all the happening events.” So, my 
interpretation of this was to send periodic control write messages 
to the device and read the responses from the interrupt endpoint. 
Also according to the documentation, the format of the write 
message is five bytes in length, so with this information, I used 
the test_libhid.c file included with libhid to see what happens. I 
found that functions within libhid give error codes if they fail and 
that the /var/log/messages file, with the extra DEBUG informa- 
tion from the modified kernel file, reports useful errors. 

Upon closer inspection of the snoop log, I saw that the 
control write was, in fact, eight bytes in length. See 
SetupPacket in snoop log output. The five bytes described 
in the documentation seemed to represent the first five bytes 
of the packet, and the last three bytes seemed to be padding. 
That is, changing these last three bytes doesn’t seem to 
affect the operation. Subsequent error-free testing, with the 
packet set to eight bytes, confirmed that the documentation 
had been misleading. 


Conclusion 

In terms of where to start with this project, I found the mailing 
list for libhid to be helpful. The libusb mailing list also provid- 
ed guidelines. The Linux usbutils are quite useful in determin- 
ing what interfaces are available on the device and the meaning 
of the descriptors. 

The libhid source code, still in constant development, also 
is a source of help. Because the code constantly is being devel- 
oped, it is a good idea to keep an eye on the Subversion reposi- 
tory for changes, including documentation changes such as 
helpful comments in the code. 
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The Only Silo 


“Edison's light bulb was important not because he was the first with the idea; as 
many as ten others envisioned similar schemes. Rather it was significant because he 
conceived not just a bulb but a whole electrified world.”—Teresa Riordan, US News 


BY DOC SEARLS 


e’ve been fighting closed and proprietary soft- 
ware for a long time now. And we’ ve had lots 
of success—enough, I think, that we need to 
move to the next stage: to the marketplace. 
We can see the problem when we look at how many closed 
systems have open foundations: Google and Amazon on 
Linux, Apple’s Mac OS X and Yahoo’s search infrastructure 
on BSD. Also, countless closed Web habitats served up 
by Apache. 

Am I being unfair here? Perhaps a little. You can’t be open 
in every possible respect, right? Some stuff needs to be locked 
down or closed off. Customer data, future product plans, trade 
secrets and “secret sauces” of one kind or another. But those 


aren’t the issue. 

The real issue is silos: closed habitats that serve as private 
marketplaces that lock customers in and competitors out. 

Dick Hardt of Sxip Networks gives a killer talk about 
“Identity 2.0”. As Dick puts it, Identity 1.0 is a province of 
walled gardens. Amazon, eBay, Flickr and Skype are all walled 
gardens. They may be lovely places to hang out, but they are 
also enclosed and private market spaces, as false in their own 
way as the faux Venetian canals and Parisian streets in Las 
Vegas hotels. 

What makes them most different from closed systems of 
the traditional sort is not a lack of interoperability—often they 
have that—but the lock-in of personal data. You can’t take 
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your eBay reputation, or the business you’ ve built inside 
eBay’s walled garden, over to Amazon. Even trivial data, such 
as your Skype contact list, isn’t portable. It’s locked inside a 
space that is not your own. 

To hear Dick describe it, Identity 1.0 is barely past 
medieval. It’s a country of duchies and city states. But since 
we’re so used to it, we can barely think outside its walls. Yet 
that’s where we belong, he says. The world we want—the 
Identity 2.0 world—is one of independent actors: free-range 
customers, conducting business and building relationships in 
ways that each individual controls and that work with many 
different vendors. 

The problem with the walled garden metaphor is that all the 
familiar examples are native to the Web. Silos, on the other 
hand, are everywhere, both on the Web and off. Nearly every 
familiar business category you can name—banking, hospitality, 
retailing, commercial aviation, car rental...even office equip- 
ment, such as copiers and printers—is a forest of silos. Take 
airlines. I am a registered frequent flyer with United, 
American, Delta, America West and Southwest. Yet the only 
common way I can relate to any of the five is money. None of 
my data in the United silo is available for my dealings with 
American or Delta. 

We’ ve lived in a world packed with silos for so long 
that we now confuse them with a free marketplace. We 


left DG and formed a new systems integration company in 
Raleigh called BAS, for Business Application Systems. 
BAS’s goal was to produce what they called “machine- 
independent software”. To be independent, Earl explained, 
you needed software that was independent of every hard- 
ware vendor’s silo. 

One day, early in my company’s relationship with BAS, 
Earl explained the ideals of machine independence, all of 
which are familiar to anybody acquainted with open source 
today. (Although, naturally, BAS’s code was proprietary.) 
When he was done with his rap, my business partner asked 
the impolite question, “So how do you make your money?” 
“We’re whores”, Earl replied. “We walk the streets with the 
rest of them.” His point: they had no choice—except 
among silos. (BAS, it turned out, bet on the wrong silo: 
Texas Instruments’ DS990s.) 

Silo was just one container metaphor kicking around in 
those days. Others were smokestacks and stovepipes. Today 
those metaphors have fallen behind silo in popularity. I sus- 
pect that’s because silos are completely contained. Unlike 
smokestacks and stovepipes, they don’t have an opening at 
the top. 

The prototypical office building is a silo of sorts. With its 
security systems, its employee and visitor badges, it comprises 
what David Weinberger calls “Fort Business”: 


The world we want—the Identity 2.0 world—is one of independent 


actors: free-range customers, conducting business and building 


relationships in ways that each individual controls and that work 


with many different vendors. 


actually believe that a choice of silos comprises all the con- 
ditions required for a free market. We can see how limited 
this is when we look at the market category we call com- 
puters. A quarter century ago, we thought the category was 
free and open because we had a choice of silos from IBM, 
Digital, Data General, Wang and HP. We thought the same 
way about networks when our choices were OmniNet, 
WangNet, IBM Token Ring, Sytek, Corvus, 3Com and 
Ungermann-Bass. 

I remember a long conversation I had with Ralph 
Ungermann about how his company’s goods were “open” 
because they interoperated with other networks. In a rela- 
tive sense, they may have been. But the market was essen- 
tially a field of silos. What he offered was inter-silo-oper- 
ability. Good for its day, but nothing like the Net that was 
to come—and which didn’t come from any one vendor 
at all. 

I remember Earl Gillmor talking about silos, way back 
in 1980. Earl enjoyed a small measure of notoriety as a 
member of a splinter group at Data General that lost “the 
shoot-out at the Holiday Inn” in Durham, North Carolina— 
an event immortalized in the early pages of Tracy Kidder’s 
book, The Soul of a New Machine. After the shoot-out, Earl 


This fort is, at its heart, a place apart. We report there every 
morning and spend the next eight, ten or twelve hours inacces- 
sible to the “real” world. The portcullis drops not only to keep 
out our enemies, but to separate us from distractions such as our 
families. As the drawbridge goes up behind us, we become 
businesspeople, different enough from our normal selves that 
when we first bring our children to the office, they’ve been 
known to hide under our desks, crying. 


Within this world, the Web looks like a medium that exists to 
allow Fort Business to publish on-line marketing materials and 
make credit-card sales easier than ever....The Web isn’t primari- 
ly a medium for information, marketing or sales. It’s a world in 
which people meet, talk, build, fight, love and play. In fact, the 
Web world is bigger than the business world and is swallowing 
the business world whole. The vague rumblings you’ re hearing 
are the sounds of digestion. 


The change is so profound that it’s not merely a negation of 
the current situation. You can’t just put a big “not” in front 
of Fort Business and say, “Ah, the walls are coming down.” 
No, the true opposite of a fort isn’t an unwalled city. It’s 

a conversation. 
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David wrote that more than six years ago, in Chapter 6 
of The Cluetrain Manifesto. We still aren’t having the con- 
versation required to bring the walls down. True, there are 
some significant conversations growing out of employee 
blogs. For example, nothing has done more to bring down 
Microsoft’s walls than interaction with outsiders by Robert 
Scoble, Kim Cameron (a subject of last month’s column) 
and about 2,000 other blogging Microsoft employees. 

But the problem isn’t communication. It’s the structure 
of markets themselves. I’m not talking about structure in an 
architectural sense, but in a deeper way that’s more like 
geology. Because the Internet is geological, not just archi- 
tectural. It has a nature that goes deeper than whatever 
structures private efforts can provide. But that nature is 
hard to see when your frames of reference are closed 
and proprietary. 

Like many in the Linux community (including my good 
friend Eric Raymond), I have strong Libertarian sensibili- 
ties. I understand the liberating advantages of private prop- 
erty to societies and their economies. Ownership matters, 
and ownership works. But we in the Free Software and 
Open Source communities also know there are some things 
that are beyond the scope of ownership and the control 
ownership naturally implies. Earth below the crust is as 
beyond the practical scope of ownership as the weather and 
the stars. Yet they provide us with services so fundamental 
we couldn’t live without them. One of those services is a 


deep and easily ignored context for property: gravity. Real 
estate would be meaningless without the gravitational pull 
provided by a mass we’ll never see. The Net’s geology is 
like that. 

I’ve written many times about the NEA nature of the Net, 
and of all free and open-source software: Nobody owns it, 
Everybody can use it and Anybody can improve it. The same 
applies to markets, and it’s time we started improving the ones 
we’ ve got, by putting silos in a context that makes clear their 
limited advantages. 

The Supreme Court missed a chance to do that with the 
Brand X case. In a 6-3 decision that was handed down on 
June 27, 2005—the same day as Grokster, which is a big 
reason why not much of a fuss was made about it—the 
Supremes upheld a 2002 FCC ruling that classified cable 
broadband as a deregulated “information service” rather 
than a “telecommunications service”. Unpacked, that means 
the cable and telephone companies can (and will) be exclu- 
sive Internet service providers. Independent ISPs like 
Brand X and Earthlink, which don’t own physical connec- 
tions to homes and businesses, are out of luck if the cable 
and phone companies want to keep captive customers 
to themselves. 

More important, the FCC’s understanding of the Internet 
achieved the stature of law with the Brand X decision. That 
understanding is basically feature-rich broadcast. It’s a concept 
of service anchored on the supply side of the highly asymmet- 
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What makes Linux so different, and so 
successful, is that it’s not designed as a silo. 


rical distribution system the FCC has governed for most of the 
last century. 

Former FCC Chairman Michael Powell, in a speech at 
the VON (Voice On the Net) conference one year ago, said, 
“To realize the innovation dream that IP communications 
promises, however, we must ensure that a willing provider 
can reach a willing consumer over the broadband connec- 
tion.” He generously acknowledged “the importance of 
consumer empowerment” and rights such as “Freedom to 
Access Content”, which he explained with “Consumers 
should have access to their choice of legal content.” 
Generous as that may have been, it was no less top-down 
than anything owned by Rupert Murdoch. Nowhere did he 
acknowledge the Net’s most profound commercial grace: 
supporting the ability of people to go into business, and to 
do business, with anybody they please, anywhere. 

Thomas Madsen-Myedal, a young serial entrepreneur in 
Denmark who hosts the delightful annual reboot conference in 
Copenhagen, recently told me he likes and appreciates Flickr— 
the on-line photo gallery phenomenon that has taken the world 
by storm (and which was built on LAMP)—but that it has 
“lock-in” issues: 


I don’t mean “a total lock-in” in the traditional BigCo IT 
sense. More like that if open data standards existed, the col- 
lective value would be in the commons—not on one photo 
sharing site. The London bombings wouldn’t be about the 
“flickr tag”, but about the “photo tag” or just the tag—which 
in my book is much more aligned with our values and the 
society we want to create. 


Thomas’ higher-level concern is that “we’re selling out on 
values of open standard and decentralization”. What Tim 
O’Reilly calls the “architecture of participation”, Thomas says, 
is turning into something that is “based on silos” in practice. 
So, he adds, “I’m gonna try and push some open standards in 
the photo sharing space to level the competition.” 

He’ll do that, he says, by “dividing what is the commons 
from what is the product. That way, thousands of photo sharing 
products can create a collective value that’s a lot greater. 
Competition will be on the product side, rather than on who 
aggregates most of the commons.” Thomas’ site, 
23people.com, is open for beta. 

In September 2005, O’Reilly put on its second Web 2.0 
conference. (Shouldn’t they call it Web 2.1?) In September 
2004, Tim O’Reilly described Web 2.0 as “the Internet as a 
platform”. Then he added: 


We heard about that idea back in the late 1990s, at the 
height of the browser wars, but that turned out to be a 
false alarm. But I believe we’re now starting the third age 
of the Internet—the first being the telnet-era command-line 
Internet, the second the Web—and the third, well, that tale 


grows in the telling. It’s about the way that open source 
and the open standards of the Web are commoditizing 
many categories of infrastructure software, driving value 
instead to the data and business processes layered on top 
of (or within) that software; it’s about the way that Web 
sites like eBay, Amazon and Google are becoming plat- 
forms with rich add-on developer communities; it’s about 
the way that network effects and data, rather than software 
APIs, are the new tools of customer lock-in; it’s about the 
way that to be successful, software today needs to work 
above the level of a single device; it’s about the way that 
the Microsofts and Intels of tomorrow are once again 
going to blindside established players because all the rules 
of business are changing. 


That was a lead-in to the Web 2.0 conference. After 
the conference, in an interview with Richard MacManus, 
Tim said: 


I actually ran a couple of panels on this at our Open Source 
convention, a year and a half or two years ago—called “Open 
Data—Do We Need a Bill of Rights for Web Services?” We had 
people from Amazon, eBay and others trying to answer that 
question: what does it mean when we’re investing our on-line 
data in these sites? Will we end up with something like the 
Open Source movement because the companies have ended up 
locking in their users? 


....But the actual data ownership is maybe less important, in 
some areas, than people think. When we talk about user- 
contributed data, we’re not just talking about my data prop- 
er (as in having your mail stored on Gmail or Yahoo! Mail 
or whatever). We’re also talking about a kind of content 
that users are contributing to a collective work. So for 
example, Amazon Reviews—people don’t really care about 
that in the same way. They’re not saying, “Oh I created that 
review and I want to be able to export it to Barnes & Noble 
as well.” They’re creating it in a particular context of 

that community. 


....Despite what I’ve said...data lock-in absolutely should be 
a concern. I believe that data lock-in of various kinds is 
going to be one of the key tools of business advantage in 
the Internet era. I think that as companies realize this, they 
will figure out how to be evil—so to speak (to use Google’s 
terminology)—and I predict that we will in fact have some 
major battles in that area. 


As I said last month, one answer is to create ways to do 
what Drummond Reed calls “Company Relationship 
Management” (or CoRM), which should look far more 
interesting and useful to companies than their own 
Customer Relation Management (CRM) systems, which by 
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nature have no view outside the company’s own silo. In 
fact, CRMs are one of the main ways companies maintain 
their silos. 

Another is to pay more attention to where the Net’s deep, 
almost geological market-making infrastructure comes from. 
It’s not from the physical cables that run to homes or from the 
“services” available exclusively from cable and phone compa- 
nies, but from the open protocols that define the Net’s environ- 
ment. It’s also not from fancy private services inside corporate 
walled gardens but from the raw building materials that make 
deploying those services so free and easy. 

Which brings us back to the L in the LAMP suite that 
makes possible the last phrase above. 

What makes Linux so different, and so successful, is that 
it’s not designed as a silo. Linux didn’t come from a silo, 
and it had no ambitions to be a silo. At one point, Linus 
talked about “world domination”, but his tongue was in his 
cheek—even if he was indulging in prophesy that would 
prove out in the long run. 

Linux was never a business. It was, and remains, a great 
way to build anything, to support anything, for anybody. 
That’s the fundamental virtue we need to fight for when we 
go to battle. 

Our battle, however, is not with the companies that use 


open code to build walled gardens and silos. Our battle is 
with the closed, top-down silo-oriented value system that 
has been with us since the dawn of the Industrial Age. It’s 
that lame old value system that prevents us from imagining 
how we can improve markets that nobody owns and any- 
body can improve. 

The best way to shed the old mentality is to embrace our 
customers and not only their money. Today the preponderance 
of inventiveness and productivity is out in the free world, in 
the hands of free-range individuals. Linus Torvalds is the 
prototypical example of one of those individuals. There are 
countless more like him, producing all kinds of goods, expressing 
all kinds of demand—much of which they are able to supply 
for themselves, as Linus did, and with the help of others, as 
the Linux community has done. 

In fact, the only silo that matters is the most fundamen- 
tal and indivisible unit in the marketplace, the individual. 
What we need is to create and support independence, 
not dependence. 

Work to free individuals, and to take advantage of what 
they do with that freedom, and you’ll have a winning strategy 
in the new marketplace we’re all making together.# 


Doc Searls is senior editor of Linux Journal. 
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FEATURE PERSONAL DESK Os? 


Fixing Web Sites with 
GreaseMonkey 


Who says “View Source” on a Web page has to be 
a read-only proposition? Re-mix your favorite Web 
sites by changing styles, adding and removing 
elements, and more. BY NIGEL MCFARLANE 


ere’s a Strange thing: hacking open source isn’t done 

only at midnight, in the spare room, hunched over the 

protocol analyser, the breadboard, source code control 

and some helpless device. No, sometimes it’s done 
inside a different crucible entirely: a public world of shameless 
posturing and self-promotion. A lurid and neon habitation of 
signs, shops, styles and stuff populated by the babble of conver- 
sations both informed and banal. It’s a place of great joy and 
great angst; a place of towering conservatism and the last bastion 
of the radical voice. Within it, a good hairdo or a radically cut 
legline can get you as far as a symbolic debugger, possibly even 
further. Devices they may be, but of a different cut entirely from 
those of hardware. Its denizens slip hyperactively in and out of 
view like character actors with coffee addictions and inspired 
agents. Of course, I refer to the World Wide Web. 

In this article, you learn how to code in a new way, a way that’s 
about changing media, not about changing programs. To enter this 
nightclub and experience the beat, you need the right gear, and the 
right gear is Mozilla Firefox and GreaseMonkey. Alfred Bester and 
William Gibson are waiting, so ready your Mojo and prepare for 
cyberspace insertion. But first, a bit of background. 


Web Pages as Open Source 

We tend to forget that the Web is open source, in a way. Some of 
the Web’s infrastructure, browsers and servers, is traditional 
open-source software, but the idea also applies to Web page con- 
tent. Appropriation of code is an everyday occurrence. Every day, 
Web developers and designers use the View Source browser fea- 
ture to appropriate (industry term: steal) code and design from 
other people’s pages. It was ever thus, and it remains so. Ideas 
and code are shared freely and often; it’s an art design sensibility. 

Most technical people have dabbled with Web develop- 
ment, and dabbling is an easy way to have a bad experience. 
The big three technologies—HTML, CSS and JavaScript— 
were riddled with bugs for many years after their inception. 
That’s the experience that probably looms large for early 
adopters who first tried it out in the 1990s and walked away in 
disgust. Cross-browser code? No, thank you. 

Fortunately, matters have improved tremendously as of late, 
and the Web is reviving as a technology platform. Better stan- 
dards support, more standards support and the decline of hoary 
old misgivings, such as Netscape Communicator 4.x and 
Internet Explorer 5.0, have left Web developers with a nearly 


clear shot at real portability, a shot frustrated only by the once 
shiny but now fairly rusty Internet Explorer. In 2005, the buzz 
is about Modern DHTML, Layout without Tables, Semantic 
Markup and Asynchronous JavaScript and XML (AJAX). 
Client-side Web development is coming back, and these are the 
things of which it’s made. This time, the Web is backed by pro- 
fessionals with formal Web training and veterans with ten years 
of experience. These people have their acts together, and it’s 
possible to say things about Web technology that are no longer 
drowned out by the static of incompatibility issues. 

Supporting and colonizing this trend is the Mozilla Firefox 
Web browser, and Mozilla technology in general. Of course, 
Mozilla is fully open source, as open as a religious movement 
can be, and so there’s plenty of room for experimentation. The 
critical bit of Mozilla and Firefox is its interpreted nature. On top 
of a big, bad, networked C++ rendering engine is a thin skin of 
JavaScript scripts and XUL, an XML dialect. This makes Mozilla 
a distant cousin to Emacs or Tel/Tk, as it provides the whole 
Firefox user interface by way of interpreted code. By writing an 
extension, you can enhance this user interface and drop it in to 
thousands of willing people’s daily experience. Go to 
update.mozilla.org to see the endless possibilities made real by 
this system. Every variant hardware device requires Linux kernel 
driver support; every variant human expectation about user inter- 
faces requires a Firefox extension. That’s a lot of extensions. 


Grabbing GreaseMonkey 

GreaseMonkey is a Firefox extension (see the on-line 
Resources). You have to click on the link twice, once to trust 
www.mozdev.org and once afterward to install the extension. 
GreaseMonkey differs from the other extensions because it 
provides no specific user-interface enhancements of its own 
other than a configuration dialog box. Instead, it creates a 
macro-like scripting environment into which you put 
JavaScript scripts. Those scripts operate on Web pages that you 
specify. When such a page loads, your script goes to work on 
the page content, no matter who provided it. You’ re intercept- 
ing a content provider’s content and modifying it before it hits 
you. No wonder GreaseMonkey’s been called “TiVo for the 
Web”. I wrote about page modification tactics in Rapid 
Application Development with Mozilla (Prentice Hall, 2004), 
but GreaseMonkey has moved that idea into the mainstream by 
supporting traditional Web-scripting techniques and by packag- 
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ing it all up into a digestible product. 

For all Firefox extensions, you must restart Firefox com- 
pletely to finish the install. Use File->Exit to do that safely. 

Bucket-loads of pre-existing GreaseMonkey scripts are 
available (see Resources). Before you get too excited though, 
note that such scripts are tied to one Firefox installation and 
have no effect on any server. On a Linux or UNIX box, such 
scripts might affect a large user population, but they’re primar- 
ily a personal thing. For those readers switched on to people 
problems, the broader implications should be obvious. 


Spotlight on Linuxjournal.com 

To see all this at work, in this article I hack the Linux Journal 
Web site with GreaseMonkey. My esteemed editor, Don Marti, 
even asked me to do this. A brave man indeed. [Maybe next 
time they'll invite me to the Web site meeting.—Ed.] 

Give me a hill and ’ll climb it. First up is a bit of scrutiny of 
the site due for surgery. Recall it’s www.linuxjournal.com, if 
you’ re reading this in print. This is also the fun part; personal 
tastes differ, and for user-side drivers—which effectively is what 
GreaseMonkey scripts are—it’s entirely valid and professional to 
be picky and subjective. In Mozilla-land, dogfood means testing 
your fixed bugs for technical correctness, and catfood means test- 
ing your inventions against unreliable and subjective people who 
might spring in any direction. It’s all catfood here, and there’s no 
right or wrong. After reading this article, LJ’s long-suffering site 
maintainer will likely glare at me venomously or perhaps change 
the site before this sees print. Design sensibilities, you see. Sorry 
mate, they made me do it. Hard-core engineers should look away; 
you might find this analysis distressing. On to the site. 

Here’s a handful of observations. 


@ The site icon, which appears in the location bar and on the 
current tab if you use tabbed browsing is dinky and unin- 
spired. Oh well. 


m™ There’s advertising everywhere. 


@ Linux Journal’s supposed to be the granddaddy of technical 
journals in open source, excluding academia and profession- 
al bodies. Where’s that indicated? 


™ The headings are red. What’s with red? I’m not in a hurry. 


On the plus side—my survival as a critic is at stake—the 
site has a robust three-column layout and is clean overall. 
Someone knows his or her stuff. Viewing the source, the layout 
is all done with CSS, so that’s relatively modern; many indus- 
try sites still pump out the worst HTML you can imagine. The 
excessive use of <DIV> tags shows that LJ is halfway through 
modernisation; there’s still some Semantic Markup work to go, 
where meaningful tags are used as content descriptors instead 
of the meaningless <DIV>. That update might improve the 
site’s search engine performance or so it’s claimed. 


The Right Tool for the Right Job 

Now, of the above personal observations, some are simple to 
rectify and do not require GreaseMonkey. If you dislike adver- 
tising, then the AdBlock extension is for you; there’s nothing, 
or at least little, to code. Similarly, for a long time, all browsers 


have supported user-specified stylesheets. If you install the 
ChromEdit extension, you can get at that stylesheet without 
having to grovel through the filesystem looking for it. Bring it 
up via Tools—Edit User Files, click the userContent.css tab 
and start typing. To make headings blue, you might add: 
hl.title a { color : blue !important; } 
@-moz-document domain(linuxjournal.com) { 

hi.title a { color 
} 


: blue !important; } 


The first rule applies to all Web sites; the second is a 
Mozilla special that applies only to the Web site specified. 
Browser-specific is okay here, because we’re working purely 
on and in the client side. 

You can get a lot done in these stylesheets, especially if you 
know CSS well. You can hack the page’s layout to bits by 
reordering, hiding or floating columns and other content. All of 
these options are possible via GreaseMonkey as well, but 
GreaseMonkey is better suited to bigger stuff. In other words, 
don’t go to GreaseMonkey if page changes are easily solved 
with a stylesheet; it’s overkill. 

For this article, we’ ll make one simple change. We’ll bring 
some gravitas to the site by replacing some content with fancy 
calligraphy drawn from another site. 


Illuminated Drop-Caps for Paragraphs 
The CSS :first-letter pseudo-selector lets you take an ordinary 
paragraph of text and make the first letter big, so that several 
lines of text flow around it. It’s a self-important feature and 
what we’re looking for. We simply could apply that feature, but 
most computers don’t have fancy medieval fonts installed. And, 
a big Times Roman letter F isn’t that exciting. It would be bet- 
ter if we could get the LinuxJournal.com Web site properly illu- 
minated, like the Book of Kells, with extra fancy calligraphy. 
Here are a couple of screenshots showing the before-and- 
after looks, taken on Windows XP Professional. This is a time- 
ly reminder that the user experience is what’s important here. It 
also emphasizes that open source means cross-platform when 
stated in Mozilla terms. Everything described in this article 
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Figure 1. A Regular Linux Site 
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Figure 2. That’s better. We didn’t need monks to illuminate this manuscript, sim- 
ply a GreaseMonkey script. 


works identically on Windows, Macintosh, Linux and various 
obscure Mozilla platforms, such as Solaris. 

In the second screenshot, you can see that the first letter of 
each paragraph has been replaced with a fancy illuminated letter. 
Because I don’t have access to the back end of the LJ Web site, 
that’s something of a feat. In fact, these images come from the 
Australian National University’s Medieval Studies image server. 

I’ve used the thumbnail images only. It’s a bit naughty to 


serve up some other Web site’s images, and these images aren’t 
perfectly cropped, registered scans, but for the purposes of, well, 
illustrating a technique, they’ll do. Let’s hope some parsimonious 
old sod doesn’t take them down by the time you read this. 


Spinning Up the Script 

To make this embellishment work, you need a GreaseMonkey 
JavaScript script. To make such a script, proceed as though this 
were any other Web page project. I saved to local disk the LJ 
home page and then added this to the end of the <head> section: 


<script src="illuminate.js></script> 


Now I’m free to develop that script in pure Mozilla JavaScript, 
with no cross-browser constraints, because GreaseMonkey works 
only on Firefox. Let me tell you, it’s real 200%-proof pleasure to 
charge forward in JavaScript without once having to trip over 
document.all or other MSIE aberations. More than that, there’s a 
bit of now-established rigor we can bring to the code. Here’s a 
skeleton of the job at hand, in the shape of a JS object signature, 
which is the bit of syntactic dogma that I like to propagate: 


var illuminate = { 


caps = { aan Fy 

load : function () { ... }, 
image : function (text) { ... }, 
mask : function () { ... }, 
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insert : function () {... }, 
getElements : function (node) { ... }, 
getElementsRecursive : function (1, n) { ... }, 


There’s no static typing and barely any forward declarations 
in JavaScript, so there’s nowhere to declare an object. Instead, 
we work with an object literal. This approach creates a pack- 
aged set of functionalities that expose only the illuminate 


Listing 1. Illuminating the Current Object 


option to the page’s namespace. So it’s both a reuse strategy 
and a namespace non-pollution strategy. All the methods of the 
object are expressed as anonymous functions, and caps is a 
sub-object in which we put data. Anonymous functions also save 
you from forcing a function name into the page’s namespace. 
That’s a vast improvement on early scripting techniques. 

Once defined, this object does nothing. You need a line 
such as this to make it go: 


var illuminate = { 


caps 2 4 
Hey 8 MOA PME Me 2 lol} PANG”. 
He 8 SHIGE PMG", TCs MilO)S PANG” 
TON NOG (PNG, Ma” NGA PING 
(2 onl ShipNGiuemetLumdmeulO Ome NGaus 
ee MN) PME", Mae me 
ikcieaen ali PNG Deel am lel aN Gale 
Or ee STS) NG, al Mae NE 
POL eee SENG es iOaeeeulelOnipNGal 
Mey 8 MA AMG, pe ok Malas) PANE 
High! 28 Mahal) (PME Pe 2 Late) PINE 
oS DULL 
TN) 8 Mala (NG 
we} nell 
Hye 8 MG) PNG 
yo nl 
PZ el? SeaRNGe 

es 


load : function () { 
this.mask(); 
this.insert(); 


Ie 


image : function(text) { 
var a = text.substring(0,1).toLowerCase() ; 
var link = ""; 
if (a & this.caps[a]l) { 
link = 'http://rubens.anu.edu.au/htdocs/' + 
‘bytype/prints/ornament/0001/' + 
this.caps[a]; 
} 


return Link; 


ye 


mask : function () { 
var head = document.getElementsByTagName( 
"head') [0]; 
var rules = document.createElement('style'); 
var text = document.createTextNode( 
"div.node > div.content > img[ill] : * + 
"t ohisykenye waitress wi@eker Ikesies A! a 


‘div, node > div, links = ° + 
clear > lett: pin’ + 


“Imelilil = £ display = mone; } ni 
Ne 
rules.appendChild(text) ; 
head. appendChild(rules) ; 


WNSQeu = WOMCHieM (Oy 
var list = this.getElements (window. document) ; 
var img; 
var text; 
for (var 1=0; i<list.length; i++) { 
text = List[i].firstChild; 
if (text.nodeType == 3 ) { 
img = document.createElement('img'); 
img.setAttribute('ill', 'true'); 
img.setAttribute('width', '64px'); 
img.setAttribute('height', '64px'); 
img.setAttribute('src', 
this. image(text.nodeValue) ) ; 
text.nodeValue = 
text.nodeValue.substring(1) ; 
List[i].insertBefore(img, text); 


getElements : function (node) { 
var rv = []; 
this.getElementsRecursive(rv, node); 
return rv; 


getElementsRecursive : function (list, node) { 
for (var i=node.childNodes.length-1;i>=0;i--) 
{ 
var child = node.childNodes.item(i); 
var klass = null; 
if ( child.nodeType == 1) { 
klass = child.getAttribute("class") ; 
if ( klass && klass == "content") { 
List.push(child) ; 
} 


this.getElementsRecursive(list, child); 
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window.onload = function () { illuminate.load(); } 


That causes the load() method to run when the page is finished 
loading. The anonymous function that wraps it provides an 
extra scope that makes illuminate the current object. Done that 
way, the reference points to the illuminate object, which means 
this can be exploited from inside the methods of the object. 
That saves the object from ever having to use the illuminate 
variable name—more namespace non-pollution. 

Listing 1 shows this object fully implemented, so let’s go 
through it. You also can grab the complete script from the 
Linux Journal FTP site. 

The caps associative array points to the individual letters 
available at the ANU Web site. Because they’re from medieval 
times, new-fangled letters such as J, U, W and Y are nowhere in 
sight. We simply have to do without those for now and also resist 
the urge to use I for J biblically. The alphabet’s constantly chang- 
ing, albeit slowly, anyway. The way I hear it, if radio is a true 
reflection of the street argot, Double-u is next to undergo change. 
Evidently it’s being replaced with an identical letter named Dub, 
as in: “Go to Dub Dub Dub dot sell you something dot com’. 
Think what you will of that despicable trend. But I digress. 

The load() method does all the work. It calls mask(), which 
inserts a <style> tag as the last thing in the head of the current 
page. Careful study of the neatly designed LJ home page lets 
one create styles that fit like extra jigsaw pieces in the existing 
layout regime. This first style acts on the new illuminated let- 
ters, allowing text to flow around them: 


div.node > div.content > img[ill] 
{ display : inline; float : left; } 

This next style stops the float effect so that the next news 
item doesn’t flow around it as well: 
div.node > div.links { clear : left; } 

That’s all standard CSS2 stuff. Finally, the rest of the 
JavaScript code is a bit over-enthusiastic in its page hacks, as 
you will see. So, here’s a style to shut up the accidental extras: 
img [ill] { display : none; } 

In all cases, the ill bit is simply a custom tag attribute 
added to identify the images specific to this script, so that they 
can be picked out easily with a style rule. 

The second thing that the load() method does is call the 
insert() method, which adds <img> tags to the main content of 
the page. To those who dabble only in client-side scripting, 
perhaps an onclick handler or two, this looks fairly formidable, 
but it’s pretty routine stuff for quality client-side scripting. 

The insert() method acquires a list of important nodes in the 
page using the now robustly supported DOM APIs. It then uses a 
loop to run through that list, adding an <img> tag of this kind: 


<img ill="true" width="64px" height="64px" src=""> 
This is added to every node found that has a Text node as its 


first child. That amounts to adding a new child node to any 
<div class="content"> tag that’s immediately inside a 


<div class="node"> tag. That’s a big assumption about the page’s 
structure. Also, there are many unwanted examples of that combi- 
nation, for instance, in the advertising column on the right and in 

miscellaneous content outside the list of articles. That’s why I had 
to shut up some images with an extra style—too many are inserted. 
It keeps the code simple to use a broad brush, though. 

While developing this, I also noticed that on one instance of 
the home page, someone had added extra <p> tags to the deck of 
one article. The deck is the lead-in remarks that draw the reader 
to the full article content. That’s a simple typo or random act of 
innovation on some editor’s part. For that one article, displayed 
in a layout marginally different to the rest, the script failed to do 
anything. At least that’s better than generating an error or an 
exception and halting. It does go to show, though, how fragile 
GreaseMonkey scripts can be if one’s not circumspect enough 
and has ignored the matter of graceful degradation, in which 
scripts melt away to NO-OPs if things go pear-shaped. Any 
assumptions made about the page’s expected structure should be 
as general and as flexible as possible. Tread lightly. 

Back in the code, insert() also uses standard JavaScript string 
operations to chop the first character off the deck’s text. So that’s 
one plain textual character gone, one image of a character added. 
Between the Web and Unicode, saying the word character with- 
out caution is to flush out in a trice all the lexicographical 
pedants lurking in the woodwork. Let them come, I say. 

The rest of the object is some routine processing leveraged 
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by the insert() method. The image() function is the easier utili- 
ty: it merely performs a dictionary look-up on the caps object, 
which is effectively an associative array. JavaScript allows lit- 
eral strings to be used as array indices and object member 
names. The retrieved filename is concatenated into a full URL 
and returned. It’s simple data-driven programming. 

The other utility is the remaining two methods, 
getElements() and getElementsRecursive(). They implement a 
standard prefix tree-walking algorithm that acts on the whole 
DOM of the page and that is wrapped up in the neater facade 
of getElements(). They are page-scanning routines and not 
overly general as there are logic tests inside specific to Linux 
Journal content. Someone should write a set of qsort(3)-like 
navigation routines so one simply can plug in a comparator 
functor or two. Probably that’s already been done, but I haven’t 
tracked down such a thing for this article. 

As the DOM tree is walked, any <div class="content"> 
nodes are appended to the list of discovered nodes. There’s no 
copying at work; it’s all nodes by reference. Walking a whole 
DOM tree is a bit ambitious. For more focused GreaseMonkey 
hacks, it’s more efficient to go straight to the page element at 
issue, perhaps with a document.getElementByld() call. When 
you’re not sure about the exact structure of the page, though, 
it’s better to grope blindly through all the content with a mini- 
mum of assumptions. How directly you proceed simply 
depends on what kind of leverage you’ re looking for. 

Now that the script is developed, all that needs to be done 
is to configure it into the GreaseMonkey extension. Recall that 
so far it has been developed on a static and locally held test 
page. That configuration task is, to be frank, a bit weird, at 
least at GreaseMonkey 0.3.3. 

To get it in place, make sure the script is named 
illuminate.user.js. Next, using Firefox with GreaseMonkey 
installed, navigate to the local directory where the script is. 
On Linux that is something like: 


file:///home/nrm/ 
On Windows it may start with: 
file:///C|/Documents%20and%20Settings/nrm/Desktop/ 


Notice the three forward slashes. The file URI scheme is 
similar to NFS or SMB and, in theory, can retrieve files located 
anywhere, for example: 


file://www.example.com/something. txt 


Omit a domain and the default is localhost, which generally 
is what you want. 

Once that directory listing appears, you should see a link 
for the illuminate.user.js file. Right-click on it (context-click on 
Mac OS X), and the magic option is revealed: “Install User 
Script ...”. Pick that, because no amount of fiddling with the 
GreaseMonkey options on the Tools menu can bring you equal 
joy. The GreaseMonkey configuration dialog box appears next, 
with the new script lodged on the left. Click Add on the right, 
and type in the Linux Journal URL, like so: 


http://www. linuxjournal.com/* 


Click OK and the script’s installed. Now it can be reached via 
the Tools menu for subsequent administration. Reload the LJ 
page and everything should work, with illuminated capitals in 
place. If not, it’s time to open the JavaScript Console and go back 
to script debugging, testing with 1.0.4 and GreaseMonkey 0.3.3. 


Illumination Postmortem 

The tale of illuminated capitals thus is told, and it’s a tale of 
content aggregation. Of course, this is but a trivial example. 
You’re not restricted to patching-in a single, grubby <img> tag, 
nor must you be so sanguine about the existing page content. 
GreaseMonkey scripts can hack the page content to bits, and 
you can stick any amount of extra content into the page, from 
any source. The redoubtable XMLHttpRequest object is avail- 
able to such scripts, and it can be used to load any content in 
the world that’s accessible by HTTP—content that then can be 
put in the current page. You also can send bits of the current 
page elsewhere with this object, but that’s another article. Here 
I’ve attempted a graceful addition to the page. 

Now you might say, “this is an exercise in folly, no one will 
see my work but me.” That, however, simply is a distribution 
problem, one solvable by many different IT deployment tech- 
niques, not the least of which is hypertext links. 

Such enhancements are not so silly either. Imagine the Web 
interface to your favourite network device, perhaps a router. 
Wouldn’t it be nice if the host status lights from the open- 
source and Web-enabled Big Brother LAN-monitoring applica- 
tion appeared next to the IP addresses of the matching hosts in 
the filtering rules in the Web pages served up by that router? At 
least then you wouldn’t be trying to fix a route for a box that’s 
not even running in the first place. You’d see a big red light 
next to the entry in the router’s configuration pages. 
GreaseMonkey is exactly the right tool for such problems, 
especially since no one has access to the source pages generat- 
ed by the router’s embedded Web server. 

Furthermore, many Web pages are busy places, full of navi- 
gation widgets and data entry fields. GreaseMonkey scripts can 
hack all that to bits, removing or adding elements to the page 
that streamline the user’s individual surfing behaviour. Don’t 
like that menu bar at the top? Hide it. Can’t remember how to 
fill in that form? Add some reminder text that floats above it. 
You get the idea. 

Finally, because GreaseMonkey is content-based, analogies 
with other content media are worth considering. If there are hit 
records and hit movies, then a hit GreaseMonkey script no 
doubt will emerge in time. What political orientation it has 
with respect to the Web site it hacks will set a very interesting 
precedent. Will it be a script that protests, deconstructs, graffi- 
tis, supports or censors the site in question? Only time will tell. 
In the meantime, GreaseMonkey is a handy tool for Web 
content that’s otherwise difficult to change. 

Resources for this article: www.linuxjournal.com/article/ 
8458.4 


Nigel McFarlane is the author of Firefox Hacks 
(O'Reilly Media) and Rapid Application Development 
with Mozilla (Prentice Hall PTR). For more informa- 

| tion about Nigel’s contributions to the Open Source 
community, see page 12. 


S8H OCTOBER 2005 WWW.LINUXJOURNAL.COM 


To 
| 


— 5 
= | a 
soe fii 
Sha: ba 
FF 
—_— 
i 
ane & : 
ba aa 


|[StORIX| 


1877-STORIX1 §=sales@storix.com 


epeti 
coe 


feesePee 
a) 
Pt 


; 
| CUTTS | 


[wiceo fen ese) I) D Tech Data 


©1999-2005 Storix, Inc. All trademarks are the property of their respective owners. 


Ls 


EXCELLENCE 
nut Alva 
ee San Francisco 200425 


aA 
win 


_dm dy CONFERENCE & EXPO 


WINNER 


FEATURE PERSONAL DESIKhOsPss 


The Linux for Kids 


Experiment 


This Linux dad got the young members of his family 
set up with educational software, art tools and games. 
BY PAUL BARRY 


hen pressed to answer truthfully, most parents 

agree that raising kids is a big experiment. In the 

December 2004 issue of LJ, Diego Betancor’s 

letter motivated me to experiment with some- 
thing ve been meaning to do for some time. Diego wanted to 
see more content in LJ aimed at kids, and his suggestion was 
the inspiration for the next phase of my child-rearing experi- 
ment: moving the kids to Linux. 

My wife Deirdre and I have three young children: Joseph, 
age nine; Aaron, age seven; and Aideen, age five—see Figure 
1. As shown in the photo, indoctrination starts early in the 
Barry household: there’s a fluffy Tux in the foreground and an 
electronic Tux on the screen. 


W 


Figure 1. The “Linux for Kids” Kids, (left-to-right) Joseph, Aideen and Aaron 


With his dad being a longtime computer geek, it came as no 
surprise when Joseph took to the computer at a young age. For 
years, our home computer was a first-generation iMac, running 
Mac OS. A great 3-D shoot’em-up game came with the iMac, 
Nanosaur, that Joseph just loves. Despite this, our household 
software policy always has been to try to ensure that any soft- 
ware brought into the house is classified as educational. 
Therefore, Joseph also has a bunch of Land Before Time and 


Zoombini titles, as well as kiddie-strategy games, such as 
Darby the Dragon. Other software includes the usual encyclo- 
pedia, dinosaur and space-exploration titles. 

Aaron is the sporty child in the house as well as the artist, 
and he has been happy to sit and play with the paint application 
integrated into ClarisWorks, the simple office suite that came 
with the iMac. Aaron also likes to play with Joseph’s software, 
as well as some of Aideen’s titles, which include Green Eggs 
and Ham, Sammy's Science House and Thinking Things. 

As long as there are a lot of bright colors and funny sound 
effects, Aideen’s happy, even though this five-year-old’s atten- 
tion span is not at all lengthy. 

As great as it is, the iMac had been showing its age for 
some time. It also has become increasingly difficult to find 
original software titles for its effectively discontinued OS ver- 
sion. Trying to upgrade to Mac OS X or any modern version of 
Linux was not an option for the iMac; it’s simply too under- 
powered. Without new titles, the kids were getting bored with 
the iMac and had been asking for a new computer. They also 
constantly bugged both me and their Mum to install various 
Windows titles on our laptops—especially the demo software 
that comes free inside various cereal packets. As Deirdre has to 
run Windows 2000 for work, her laptop was the one infected 
with a growing collection of these types of titles. 

A few months back, a new computer arrived in the form of 
a Dell Optiplex GX270, with 512MB of RAM, a 40GB hard 
disk and a flat-panel monitor. As I’d rather eat the new PC than 
allow the kids to use the factory-installed Windows XP, I 
looked for a family-friendly Linux distribution to install 
instead. Having recently experimented with Ubuntu Linux as 
my office desktop, I downloaded and burned a copy of the 
Warty Warthog release for use at home. 

Like most big kids, I love experiments, and now my experi- 
ment had a plan: replace the Dell’s factory-installed OS with 
Ubuntu, pack it full of kid-friendly software, let the kids at it 
and see how they get on. 


Going Cold Turkey 

I deliberately decided against installing any type of emulation 
that would have allowed the kids to run any of their existing 
software titles, even though such technology is well established 
within the Linux community. My main reason for doing this 
was to see if the kids would identify any titles that they missed. 
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If they did, Id try to find native alternatives, install them and 
see if the yearning subsided. 


Ubuntu installed easily on the Dell, taking about one hour from 
start to finish. Once the base OS was up and running, I 
installed a bunch of stuff for the kids to use. I created a user ID 
called kids with a password of dinosaur and then set up a win- 
dow in Nautilus to mimic the look and feel of the Mac OS 
Launcher program, as shown in Figure 1. Nautilus hopefully 
would provide a familiar look and feel for my pint-sized, Mac- 
loving user community. 


Bile Edit View Places Help 
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Figure 2. The Linux for Kids Launcher, Courtesy of Nautilus 


In an attempt to ease the introduction of a new—and somewhat 
different—computer into the house, we decided to relax our 
household software policy and install a few nice Linux games 
along with the educational software. Here’s a quick rundown of 
the titles we decided to make available on the desktop launch- 
er. Unless stated otherwise, these titles were downloaded into 
Ubuntu using the included Synaptic Package Manager. It helps 
to refer to Figure 2 while working through this list. 


AisleRiot Solitaire (/usr/games/sol) is a Linux version of the 
classic solitaire game. It came pre-installed on Ubuntu and 
was elevated to the Launcher in an attempt to provide a 
familiar piece of software on the new desktop. 


Bug Squish (/ast/games/bugsquish) is a bit mindless but fun 
all the same. Little bugs drop down and try to land on an 
arm. Your mission—should you accept it—is to squish as 
many bugs as you can by clicking your mouse on them. As I 
said, it’s mindless, but it does allow little people to practice 
their mouse skills while having some fun. 


Calculator (/usr/bin/gcalctool) is the GNOME calculator. 
Four-in-a-Row/Connect 4 (/usr/games/gnect) is just like the 
board game. You can play against another human opponent 


or an increasingly more skillful computer user. 


G Compris (/usr/games/gcompris) has to be the real find of 
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the experiment. This is a single program that has many, 
many parts. It is an entire suite of educational tools pack- 
aged together and aimed at 3-8 year olds. Within the suite 
are—among many other things—word and number games, 
color-matching and memory exercises and geography 
quizzes. There are loads of educational functionality in G 
Compris, and it is graduated, which means children cannot 
proceed to a later exercise until they have mastered the ear- 
lier ones. G Compris can be installed in one of a number of 
languages and has a friendly soundtrack and voice-over. I 
initially thought Aaron and Aideen would spend a lot of 
time in this program and was surprised to find Joseph enjoy- 
ing it too. There’s so much in G Compris that it really needs 
to be experienced to be believed. 


K Tuberling, the Potato Guy (/usr/games/ktuberling) is a 
simple little program that provides a blank picture upon 
which you can place, for example, ears, eyes, noses, specta- 
cles, hats and hair. The default blank picture is a potato, but 
a blank Tux also is provided. Aideen loves this program, as 
do the boys. The boys love it so much that they used K 
Tuberling to create a gallery of Tux and his family. Check 
out Tux’s mother-in-law, as shown in Figure 3. 


MathWar (/ustr/bin/mathwar) is a simple X-based math-drill 
program. 


Office Draw (/usr/bin/oodraw) and Office Writer 
(/usr/bin/oowriter), both part of the OpenOffice.org suite, 
were included primarily for Aaron, who likes to draw 
with the computer as well as write short stories and 
poems. I’d recently convinced the kids’ schoolteacher to 
try OpenOffice.org for Windows in their school, in an 
effort to fix file format-compatibility problems she was 
having with the school’s existing choice of office suite. 
So, making OpenOffice.org available on the kids’ PC 
made perfect sense. 


Play a DVD (/usr/bin/xine) allows the kids to view any of 
the DVDs that they own. To get DVD playing to work on 
Ubuntu, I had to search Google for the libdvdcss library, 
which allows for the DVD movie encoding to be deci- 
phered. Once the library was installed, DVD viewing 
worked. Xine was a big hit, not only because it supports 
DVD menus and the like but also because it allows viewers 
to capture snapshots of the currently playing movie. Once 
he discovered this Xine feature, Joseph wasted no time and 
created a gallery of snapshots of his current DVD favorite, 
The Incredibles. An added bonus to being able to view 
DVDs on the new computer is that the main household TV 
and DVD player are freed-up for Mum and Dad to use. Xine 
was chosen over the Ubuntu-installed Totem, which did not 
work as well as Xine in any of my tests. 


Play a Music CD (/usr/bin/gnome-cd) turns the PC into a 
CD player, with the default GNOME CD player popping up 
whenever an audio CD is popped in to the CD drive. 


@ Super Tux (/usr/games/supertux) is a classic, Mario-style, 


jump-and-bump-level game that should be familiar to many 


readers. Saying that the boys love this game would be a 
complete understatement: they are totally besotted with it. A 
little animated Penguin jumps and bumps his way through 
increasingly difficult levels in search of his goal. The sound- 
track to this game is great, as are the effects and configura- 
bility. If anything, it’s a little too addictive and, of all the 
programs described in this article, Super Tux is the program 
most likely to be on-screen when I enter the playroom. This 
has caused Deirdre to worry that the boys are playing it too 
much. However, as the game allows players to design and 
use their own levels, and as the boys have started to do just 
that, ve been happy to let Super Tux survive. I figure that 
building a level is the first tentative step toward getting the 
computer to work the way the kids want it to, which isn’t a 
huge leap away from that other popular customization tech- 
nique: programming. So, highly addictive or not, Super Tux 
stays for now—unless the boys are cheeky to their Mum, in 
which case it'll be wiped from the PC faster than they can 
say “yahtzee!” 


Tali/Yahtzee (/usr/games/gtali) is a nice implementation of 
the classic dice game. The iMac had a great version of this 
game that the boys always liked to play, and the GNOME 

version is similar and familiar. 


Tux Kart (/usr/games/tuxkart) is an arcade-type racer game. 
Little Tux sits in a go-kart and races around one of a selec- 
tion of pre-built tracks. The music is fun, and the game is 
not too hard to play, which means that even Aideen can play 
without too much trouble. ve seen some games of this type 
that take the physics to the extreme, making them incredibly 
hard to play well. Tux Kart, thankfully, does not fall into 
this category. 


Tux Paint (/ust/bin/tuxpaint) is a great kids-targeted drawing 
program. The sound is great, the effects are wonderful and it 
is easy to use. Aideen spends more time in Tux Paint than in 
all of the other installed programs combined, and Aaron 
enjoys using it too. The built-in collection of stamper shapes 
especially are appreciated by our budding Picassos. 


Tux Racer (/usr/games/tuxracer) is the one program that’s 
fired-up and shown-off whenever either of the boys have a 
friend over to play. Tux Racer is, quite simply, one very cool 
program. Watching Tux slide on his belly at 90km/h in 
stunning, realistically rendered graphics remains—for me, 
anyway—one of the best examples of just how far Linux 
has come as a multimedia platform. 


Tux Type (/usr/games/tuxtype) is a fun typing tutor. All three 
of the kids play it, and Aideen loves the way Tux eats the 
letters as they drop from the sky and correctly are identified 
on the keyboard. Aideen especially likes the cartoon-type 
sound effects and animation that occur when Tux eats a fish 
at the last possible moment, which usually results in Tux 
making a mad dash across the screen. 


X Tux (/ust/games/xtux) is a 2-D, Pac Man-type game that 
works well and is fun to play. Although not as popular with the 
boys as Tux Racer or Super Tux, it still is played quite often. 
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Figure 3. A Rather Cool-Looking Granny Tux 


Problems 

Thankfully, there are no show-stopper problems to report. 
The Warty Warthog release of Ubuntu did have some prob- 
lems with sound. After a restart or a new login, the sound 
configuration would be lost, resulting in no more sound. 
Upon investigation, I discovered that the GNOME volume 
controls were being set automatically to zero. To fix this 
temporarily, I popped a shortcut to the GNOME Volume 
Control applet on the desktop and then used it to reset all 
the volume sliders. This fixed the sound problem, until the 
next restart or login, of course. I planned to research a per- 
manent fix but then quickly realized that the complaints 
about the new computer having no sound had stopped. It 
turned out that Aaron had watched me fiddle with the vol- 
ume controls, he’d told his siblings what to do, and all three 
of them had developed the habit of sliding up the volume 
controls immediately after logging in. 


Upgrading Ubuntu 
In the last few weeks, I upgraded the PC to the most recent 
release of Ubuntu, Hoary Hedgehog. This resulted in much 
merriment, primarily because of the inclusion of a newer 
release of Super Tux that, I’m told, is much better, has 
improved graphics, animation and sound. Speaking of 
sound, this Ubuntu release is better but still has a few prob- 
lems. Any that surfaced were all fixable, permanently, and 
all I needed to do was search the Ubuntu support wiki for 
sound and the name of the program that was misbehaving. 
The fixes found in the wiki worked, and sound is no longer 
a problem. 

With the upgrade, Joseph asked if the shared login ID 
could be replaced by individual IDs, which I did. This is 
less to do with privacy and more to do with his little sis- 


ter’s fondness for pressing the Delete key when viewing 
Joseph’s K Tuberling Tux family collections. By the way, 
Tux’s family has been extended to include cousins, friends 
and neighbors. 

Once the novelty factor started to wear off, I began to 
get requests for some of their older software titles. Most of 
these, despite being targeted to Mac OS, did come in dual- 
install format, in that they can be installed on Windows too. 
In an effort to see how much work was involved, I decided 
to play around with Wine in an attempt to install some of 
the titles the kids were asking for. After a few hours of 
research on the Internet and some reading, I spent about a 
day trying to get the latest release of Wine to work on 
Ubuntu. I managed to run the installers successfully for a 
lot of the Windows titles that the kids had, but none of the 
programs would run properly once installed, so I had to 
abandon the effort. Since giving up—and since the upgrade 
to the latest Ubuntu—the requests for the older titles have 
become less frequent; although Aaron misses one of the 
freebie, cereal-pack soccer games that he used to play on 
his Mum’s laptop. As I finish off this article, I’m in the 
process of downloading and evaluating a small collection 
of Linux soccer games from The Linux Game Tome. The 
Eat The Whistle technology looks the most promising. If 
this does not satisfy Aaron’s craving for a soccer game, I 
plan to dedicate additional time to configuring Wine. 


Is Linux Ready for Kids? 

The answer is yes, of course it is! It’s not that Linux is a 
better platform than the others for kids to use, it’s that 
Linux is as good as any other. Children are happy to sit 
down and play with most any computer as long as the soft- 
ware titles provided are engaging and fun. This is true of 
Linux, Windows and Mac OS. Of course, the point to make 
is that if Linux is as good as the others, there’s nothing 
stopping anyone from using Linux as a primary OS for chil- 
dren. It’s not a case of “is Linux ready for kids?” but rather 
“why not Linux for kids?” 

The Barry household has made the move to Linux and 
won't be turning back. The wealth of software available on 
the Internet and within Ubuntu’s Debian archives has been 
only scratched. There are loads out there for me to evaluate 
and install for my kids as they grow out of the programs 
they currently are enjoying. If you have any suggestions for 
programs you think they might like, drop me a note and 
we'll take a look. 


Acknowledgement 
Thanks to Peter Garrett from Marcel Gagné’s WFTL-LUG 

mailing list for suggesting I use Nautilus to mimic the Mac OS 
Launcher application.# 
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Why its LPI the Global Standard in 
Linux Certification? 


All Linux Professional Institute certification programs are created using extensive 
community input, combined with rigorous psychometric scrutiny and professional 
delivery. We test the whole continuum of important Linux skills - we don't just focus on 
small, subjective tasks. LPI exams are not simply an afterthought used to help sell 
something else. LPI is a non-profit group that does not sell software, training or books. 
Our programs and policies are designed to meet educational requirements, not 


ible. 


LPI exams are available in seven languages, at more than 7,000 locations, in more than 
100 countries. You take LPI exams when you want, where you want. In addition, special 
exam lab events around the world make our program even more affordable. And 
because we don't make exclusive partnerships, LPI is supported by a broad range of 
testing centers, book publishers and innovative suppliers of preparation materials. 


You switched to Linux to get away from single-vendor dependence. So why trade one 
form of vendor lock-in for another? LPI's program follows the LSB specification, so 
people who pass our tests can work on all major distributions. Because of its strong 
grass-roots base and corporate support both inside and outside the world of open 
source, LPI goes beyond "vendor-neutral" to truly address community needs. 


LPl is IT certification done RIGHT! 


For more information, please contact us at Linux 
Info@lpi.org or visit us at Professional 
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Project Utopia 


Users—what will they plug in next? Robert is 
making the computer make sense of hardware, 
so you don't have to. BY ROBERT LOVE 


or the better part of a decade, Linux enthusiasts have 

waxed poetic on the inherent greatness and looming 

success of Desktop Linux. Our kernel is so robust! Our 

applications are infinitely configurable! This is the year 
of Desktop Linux! Who would ever use Microsoft Windows? 
These claims and similar—particularly when made back in the 
20th century—seem, in retrospect, so trite. Not that I stand 
righteous—oh no, I laid the praise down as thick as anyone 
else did. I too was a convert. 

At least, I did until I realized that hardware support in 
Linux was awful. Like a deck of cards, my rosy view of 
Desktop Linux came crashing down, making a 180-degree turn 
from glowing to ghastly. Heartbroken, I cried myself to sleep 
every night and went on an inexplicable diet consisting only of 
cheese and pudding. 

But this did not last long. One day, the Linux community 
decided to do something about it. We outlined a plan not only 
to reach feature parity with the other desktop operating sys- 
tems, but also to surpass them. Hardware support has come a 
long way in the last year and a half. This is the story of just 
how far. 


A Past Since Forgotten 
The steps for installing a new hardware peripheral on a Mac 
might go a bit like this: 


@ Step 1: plug hardware in to Mac. 
®@ Step 2: begin using hardware. 


Most of us would not even include these two items as steps. 
The first is a physical necessity; the second is the original and 
ultimate goal. Lost, somewhere between steps one and two, are 
39 other steps, right? Kernel modules? Configuration files? 
Rebooting? Extensive mastery of sed and awk? 

At some point in Linux’s history, support for new hard- 
ware could easily require compiling a new kernel module, 
becoming root, editing configuration files, loading said 
module, checking dmesg, cursing, removing the module, 
unplugging the hardware, plugging the hardware back in, 
reloading the module and so on. 

Forgotten, perhaps clouded by a love for free software 


and the invigoration of do-it-yourself, is the notion that 
stuff should just work. As fun as writing my own kernel 
module might be—and I use the term fun here loosely— 
sometimes I just want to plug in my camera, get my photos 
and be done with it. 


A Call for Change 

In late 2003, the Linux system was well primed for the 
emergence of a new architecture for managing hardware on 
the desktop. The 2.6 Linux kernel was out and rapidly gain- 
ing adoption. It brought, among numerous other new fea- 
tures and improvements, a new mechanism for handling 
device drivers, called the device model. The device model 
allowed, for the first time, the kernel to build an in-memory 
tree of the devices it supported. For example, both my 
mouse and my keyboard are connected to my USB hub, 
which is connected to my third USB port, which is on my 
first PCI bus. Such a rich hierarchy provides all sorts of 
opportunities to the kernel. One of the most promising, 
however, was sysfs. 

sysfs exports this device hierarchy as a filesystem. One 
directory lists all the buses on a system. For each bus, another 
directory lists all the devices on a given bus. Files for a given 
device could link to the associated module. Walking the sysfs 
tree, therefore, would allow user space to build a comprehen- 
sive picture of the system’s physical device hierarchy, exactly 
as the kernel sees it. 

That alone is incredibly useful. But another kernel feature, 
hotplug, broadened the horizon even more. The kernel’s hot- 
plug infrastructure notifies user space whenever a device is 
added to or removed from the system. This allows applications 
to become aware of changes to sysfs in real time. It also 
allowed for the creation of udev. 

udev is a user-space implementation of devfs—an auto- 
mated and dynamic manager of device nodes. Instead of a 
/dev created once, statically, udev updates /dev on the fly, 
in response to the exact hardware available to the system. 
More important, however, is that udev places intimate 
knowledge of devices and their device nodes in user space. 
Hotplug, sysfs and udev together allow user space a com- 
plete view of the system’s hardware. 

Now user space needed to capitalize on the opportunity. 
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Figure 1. Early Project Utopia Notes 


with a single, comprehensive view of hardware, accessible 
via a standardized interface. HAL makes it possible for an 
application to say, “give me the device nodes of all input 
devices” or to ask, “is there a camera connected to this com- 
puter?” With HAL, what was once a hundred lines of hacks, 
operating on hard-coded device nodes with intimate knowl- 
edge of Linux internals, is now a single, elegant HAL 
request. David’s HAL, in effect, brought a 21st-century 
hardware infrastructure to Linux. 

The second disruptive event came in December of the 
same year, when I accepted a job with Ximian, recently 
acquired by Novell, as a kernel hacker dedicated to the 
desktop. My first mission was to figure out the hardware 
situation. I teamed up with a colleague, the inimitable Joey 
Shaw, an Ohio native, and we sat down and hashed out our 
utopian view of hardware management. 

Both Joey and I recognized the strong foundation that the 
2.6 Linux kernel, sysfs, hotplug, udev and now HAL supplied. 
We concluded that the missing pieces were the layers on top of 
HAL. We had a rich infrastructure in place; we just had to do 
something with it. 

HAL uses a then-nascent but always-promising project 
called D-BUS as its communications mechanism. On one 
side, D-BUS is a run-of-the-mill interprocess communica- 


Enter HAL 

This was 2003. That summer, I 
attended a BOF at the Ottawa Linux 
Symposium on improving the Linux 
desktop by Robert Sanford Havoc 
Pennington. In the BOF, Havoc refer- 
enced a whitepaper of his entitled 
“Making Hardware Just Work”, in 
which he unveiled a utopian view 
of hardware management on the 
Linux desktop. Intrigued, I took 
notes—see Figure 1. 

We ended up speaking to the group 
on this utopia and discussing possible 
implementations. The BOF ended with- 
out much traction from the audience, 
but Havoc and I had a firm understand- 
ing of the situation and potential solu- 
tions. Other responsibilities kept me 
from immediately acting on my crude 
sketches, and so they sat idle on the 
pages of my notebook. 

Two things happened that lifted the 
pages to life without my immediate 
realization. 

First, David Zeuthen, then living 
in Copenhagen, decided to bring 
Havoc’s documents to life by begin- 
ning the HAL Project. HAL, original- 
ly hardware abstraction layer but now 
not an abstraction of anything what- 
soever, is a system-level demon that 
ties together hotplug, sysfs and udev 
in order to provide a Linux system 


tion (IPC) system—like CORBA, but a lot easier to use. On 
the other side, however, D-BUS introduces the concept of 
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the system-wide message bus. In addition to per-user pro- 
cess-to-process communication, D-BUS allows components 
in a Linux system to send out signals, announcing events or 
providing information to all who care to listen. Signals can 
announce when a network connection is obtained or when 
the laptop battery is running low. Interested applications 
higher up the stack can listen for these signals and, upon 
receipt, react. 

Our plan was literally to flood the system with D-BUS sig- 
nals. HAL and other lower-level components of the Linux sys- 
tem were to generate numerous useful signals and have higher- 
level components respond, evolve and react. In effect, our goal 
was to make the Linux system much more dynamic and, ulti- 
mately, make hardware just work. 


A Project All about Utopia 

Joey and I decided to create an umbrella project—a meta-pro- 
ject. The plan was to spur development of HAL-aware applica- 
tions that can provide hardware policy on the desktop. Never 
should a user need to configure hardware. It should happen 
automatically in response to the user plugging the hardware in. 
Never should the user (or even the programmer) have to mess 
with device nodes and esoteric settings. HAL should provide 
all of that, on the fly, to the applications. Never should the user 
have to guess how to use new hardware. If I plug in a camera, 
my photo application should run. If I insert a DVD, it should 
start playing. All of this should happen magically, automatically 
and cleanly. 

I coined the name Project Utopia. It was, after all, a 
bit utopian. 

We did not have a central Web site or source repository or 
cute logo. Project Utopia was a cause and a way of thinking. 
We had a goal and a set of use cases and a growing disgust 
toward things not working. We blogged and spoke at confer- 
ences and wrote code. One by one, piece by piece, we started 
to build a set of policy pieces on top of HAL, guided by the 
following rules: 


@ Make hardware just work. 
m™ Use HAL, udev, sysfs and 2.6 Linux kernel as our base. 
@ Tie it all together with D-BUS. 


@ No polling, no hacks—everything should be event-driven 
and automatic. 


™ Carefully divide infrastructure into system and user level. 


™ System level should be platform-agnostic; user level, 
GNOME-based. 


GNOME Volume Manager 

I began writing GNOME Volume Manager in late December 
2003. It was originally a proof of concept—a test bed for 
my ideas. I wanted to see how feasible hardware manage- 
ment on top of HAL could be. The plan was to respond to 
events such as “new hardware” or “audio CD inserted” with 
specific actions. GNOME Volume Manager is nothing but a 
simple finite state machine, receiving hardware-related 


Question 


You have new photographs on this device. 


Would you like to import these photographs into your photo 
album? 


Figure 2. GNOME Volume Manager Prompting on Discovery of New Photos 
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Figure 3. F-Spot, a Photo Management Application 


events on one end and replying with hardware-induced 
actions on the other. The tricky part was to do it all with 
HAL: no polling, no hacks. 

GNOME Volume Manager implemented the Project Utopia 
policy related to block devices. When the user inserted an 
audio CD, GNOME Volume Manager would play it. When the 
user inserted a USB keychain device, GNOME Volume 
Manager would mount it and open a Nautilus window. When 
the user plugged in a camera, GNOME Volume Manager 
would ask if it should automatically import the photos into the 
user’s photo management application (Figures 2 and 3). A 
recently added feature even found GNOME Volume Manager 
managing iPods! 


The Rest of the Puzzle 

The next step was bringing HAL support to more applica- 
tions, a process Joey and I call halification. The following 
months witnessed additional policy pieces, such as automatic 
printer configuration and seamless network management 
(Figure 4). 

For printers, Joey wrote a HAL back end for CUPS, the 
Common UNIX Printing System, allowing CUPS to query 
HAL on the availability of printers. The result: plug in a printer 
and configure it automatically, on the spot. 

The ambitious NetworkManager Project, started by 
hackers at Red Hat, aimed to solve networking woes. Seth 
Nickell, an early designer on the project, described the 
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Figure 4. NetworkManager’s Network-Switching Applet 
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Figure 5. NetworkManager Architecture 


intended use case as an electrical outlet: “you plug it in and 
[it’s] on.” For example, plug a laptop in to a docking sta- 
tion, and it instantly switches to the station’s Ethernet. 
Walk into your favorite coffee shop and instantly begin 
using the wireless networking. NetworkManager made 
networking simple, automatically choosing the optimal 
solution for networking connectivity. 

NetworkManager’s architecture is two-part. First, a root- 
level demon sits alongside HAL, responding to HAL events 
and communicating with the system’s networking hardware. 
Second, one or more user-level components implement policy 
and provide a user interface. Together, the components provide 
a complete solution for networking. Figure 5 is a diagram of 
the architecture. 


Today 

Today, the Project Utopia mindset continues to foster new 
applications, interesting hacks and fresh projects aimed at 
making hardware just work. Linux distributions from 
Novell, Red Hat and others sport powerful HAL-based 
infrastructures. The GNOME Project is integrating HAL 


and D-BUS across the board. The Project Utopia cause is 
spreading beyond GNOME too, as other platforms imple- 
ment HAL-based solutions in a similar vein. 

Linux development has never stood still, however. Like 
a rabid cheetah, development sprints forward toward better, 
faster, simpler solutions. Support for new hardware contin- 
ues to roll in, and solutions in the spirit of Project Utopia 
are continually implemented to provide a seamless user 
experience. 

Cute hacks such as having your music player mute when 
your Bluetooth-enabled cell phone receives a call are not a 
dream but the reality in which we live. What cute hacks will 
tomorrow bring? What new hardware will we support next? 
What application will be halified next? Join in and answer 
those questions yourself! 

Resources for this article: www.linuxjournal.com/article/ 
8459.8 


Robert Love is a kernel hacker in Novell’s Ximian 
Desktop group and the author of Linux Kernel 
Development (SAMS 2005), now in its second 
edition. He holds degrees in CS and Mathematics 
from the University of Florida. Robert lives in 
Cambridge, Massachusetts. 
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NEW PRODUCTS 


Supermicro SUPER PDSG4 and 
SUPER PDSGE 


Supermicro Computer announced the launch 
of Intel dual-core products, the SUPER 
PDSG4 and SUPER PDSGE motherboards, 
which support PCI-X 133/100 expansion 
cards. Based on the Intel 955X chipset, the 
SUPER PDSG4 ATX form factor board sup- 
ports one Pentium Processor Extreme Edition, 
featuring two processing cores with a 
1066/800/533MHz system bus. It also offers 
8GB of ECC unbuffered DDR2-667/533/400 
SDRAM; a user overclock feature in the sys- 
tem BIOS; PCI-Express x16/x1; three 32-bit 
PCI, two PCI-X 133/100 and four SATA ports 
(3Gbps); RAID 0, 1, 10 and 5; eight USB 2.0 
ports; onboard AC97 audio; single PCI- 
Express Gigabit LAN; and U320 single-chan- 
nel SCSI. The SUPER PDSGE is based on the 
945G/P Express and supports one Pentium D 
processor, featuring two processing cores with 
a 1066/800/533MHz system bus; 4GB of 
unbuffered DDR2-667/533/400 SDRAM; a 
user overclock feature in the system BIOS; 
PCI-Express 1x16/2x1; four 32-bit PCI and 
four SATA ports (3Gbps); eight USB 2.0 ports; 
onboard AC97 audio; single PCI-Express 
Gigabit LAN; and integrated Gfx graphics. 
Both boards are RoHS-compliant, lead-free 
and optimized for Supermicro’s SC733T-645 
and $C733i-645 mid-tower chassis. 
CONTACT Supermicro Computer, Inc., 980 


Rock Avenue, San Jose, California 95131, 408- 
503-8000, www.supermicro.com. 


PetaBox 
eee 


Capricorn Technologies introduced the PetaBox 
Product Family, designed for massive data stor- 
age. The PetaBox supports petabyte-class stor- 


age with state-of-the-art density, low power 
consumption and a low total cost of ownership. 
The PetaBox is scalable from individual tera- 
byte nodes to a full petabyte cluster. A single 
19-inch rack can support up to 64TB of raw 
disk space, a density achieved through a design 
that consumes as little as 50 watts per terabyte. 
Four models of PetaBox currently are avail- 
able: the GB1000, a 1.0TB node; the GB1600, 
a 1.6TB node; the TB40, a 40TB rack; and the 
TB64, a 64TB rack. Each node has four hard 
drives per node, an ATA interface, rotational 
vibration compensation, 8MB of cache, 8.5ms 
of typical latency and an EZ-Latch disk mount- 
ing system. Nodes also feature a 1GHz VIA C3 
CPU, up to 1GB of DDR266 RAM, two USB 
2.0 ports, 10/100 or 10/100/1000 Ethernet and 
an optional 16x2 character LCD. 

CONTACT Capricorn Technologies, 1021 


Mission Street, San Francisco, California 94103, 
415-722-2149, www.capricorn-tech.com. 
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Trolltech released ver- 
sion 4 of its Qt 
cross-platform 
development 
software. New 
~~ features for Qat4 
include improved 
heavy-duty graph- 
ics capabilities. Qt’s 
painter now supports 
semi-transparency, anti-aliasing, optional float- 
ing-point coordinate system, painter paths and 
gradients. Support for interchangeable underly- 
ing paint engines and off-screen rendering also 
has been added. Trolltech also extended Qt’s 
multithreading capabilities, along with its 
database integration and XML support for 
building both desktop and server-side applica- 
tions. In addition, Qt 4 offers seamless integra- 
tion with Microsoft Visual Studio .NET, allow- 
ing Visual Studio .NET developers to create 
applications that can run on Linux, Mac OS and 
other desktop platforms. Furthermore, Qt 4 for 
Microsoft Windows is available under the GPL. 
Three editions of Qt 4 are available: Qt 
Console, Qt Desktop and Qt Desktop Light. 
CONTACT Trolltech, Inc., 1860 Embarcadero 


Road, Suite 100, Palo Alto, California 94303, 
650-813-1676, www.trolltech.com. 


Equalizer SI Series 
i 
Coyote Point recently introduced a new line 
of Web server performance appliances 
designed to address core availability and per- 
formance requirements of Web sites and server 
farms. The Equalizer SI Series systems inte- 


grate an enhanced version of Coyote Point’s 
adaptive server load-balancing and traffic man- 
agement software, consolidated switch intelli- 
gence and Intel processor-based performance. 
Three systems are available: the enterprise- 
class E450si, the mid-range E350si and the 
entry-level E250si. Their features include con- 
solidated switching capacities of up to 16 ports; 
support for up to 8,000,000 concurrent connec- 
tions; incrementally scalable load balancing 
and traffic management for an unlimited num- 
ber of virtual servers and up to 64 servers per 
cluster; adaptive protection against DoS 
attacks; built-in Flash memory for zero-down- 
time reliability; SSL acceleration for up to 
4,000 encrypted transactions per second; and 
an enhanced Web interface for point-and-click 
operation of configuration options. 
CONTACT Coyote Point Systems, Inc., 675 
North First Street, Suite 975, San Jose, California 
95112. 


Motorola E895 


Motorola announced the release of the E895 
multimedia clam-shell handset, 
built on Motorola’s EDGE technol- 
ogy, Linux and Java. The E895 
offers a suite of intuitive multime- 
dia tools, including a 1.3 megapixel 
camera, video record and playback 
options and an optional removable 
memory. E895 features include 
3GPP video streaming and the abil- 
ity to view files as they download, Bluetooth 
wireless technology and Bluetooth Stereo 
Headset, SyncML, streaming audio, shared 
media player with multiple audio codecs and 
stereo through the enhanced mini-USB headset 
jack, up to 1OMB of embedded memory, and 
TransFlash removable memory for up to 
512MB of optional memory. The E895 also 
offers a full HTML/XHTML Web browser, 
multimedia messaging service (MMS) and 
instant messaging. 


CONTACT Motorola, Inc., 
www.motorola.com/motoinfo.& 


Please send information about releases of Linux-related 
products to Heather Mead at newproducts@ssc.com or 
New Products ¢/o Linux Journal, PO Box 55549, Seattle, 
WA 98155-0549. Submissions are edited for length 


and content. 
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by Ralf Hildebrandt and Patrick Koetter 


No Starch Press, 2005 | 


Anyone who has tried to run an e- 
mail server knows that mail isn’t a 
polite relay race anymore. It’s a game 
of smash-mouth football. Nine out of 
ten times someone opens an SMTP 
connection to you, it’s not with some- 
thing you want. And as if coping with 
spammers, viruses and other people’s 
misconfigured mail software wasn’t 
enough, now e-mail is a mission-criti- 
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cal company IT service and is expect- 
ed to plug in to the LDAP directory. 
We can’t blame you if you decide to 
outsource mail entirely. 

If you do decide to stay on as 
postmaster@ and fight it out, whatev- 
er you do, don’t try it with one of last 
decade’s mail books. Although any of 
the current mail servers, correctly 
configured, can put up a good fight 
against the spammers and other bad 
people, The Book of Postfix by itself 
is a good reason to make Postfix your 
mail server of choice. Look here for a 
good explanation of the SMTP proto- 
col, essential for any mail admin, 
along with enough detail on the archi- 
tecture of Postfix to help you really 
understand the config files. It also 
offers real-world advice for putting 
together a mail server setup that is 
reliable in the face of the spam and 
virus blitz. 

Postfix offers you a lot of choice in 
where to add filters, sanity checks and 


other protective countermeasures to 

your mail server. For example, do you 
want to set up a content_filter or an 
smtpd_proxy_filter? Besides offering a 
cookbook for each solution, The Book of 
Postfix helps you consider the pros and 
cons of each feature you’ re considering. 
A helpful plus is diagrams illustrating 
exactly where countermeasures fit into 
the Postfix architecture. 

Postfix is complicated enough on its 
own, as it divides functionality among 
multiple processes for security. In order 
to add spam-fighting tools and have 
everything work, you need a good 
understanding of what plugs in to what 
and how, and this book is a great way to 
get it. 

Downloadable scripts and errata, 
some of which could save you a late 
night of troubleshooting, are available at 
www.postfix-book.com. 


—DON MARTI 
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TEN YEARS AGO IN LJ 


The 
| October 
"| 1995 issue 

| Text Processing: EAS eons 
bate nwt AA od ma ue Text 
j # Processing”, 
and feature 
articles 
: —38 introduced 

’ groff, 
We} + LaTeX 

— and 
Linuxdoc-SGML, which was an 
early document format at the Linux 
Documentation Project. All three 
document formats are still in 


October 1995 


use today. 

Making the transition to 64 
bits is IT news today, but it was 
a hot topic for us ten years ago. 
Jon “maddog” Hall, then still at 
Digital, covered Linux on Alpha 
and its advantages for computer 
science education: 


Over time, this meant that to get 
all the sources to our Unix prod- 
ucts, 15 separate licenses were 
necessary, at a cost of thousands 
of dollars, and even then the 
sources were restricted to a “need 
to know” basis and were not for 


consumption by curious students. 


Publisher Phil Hughes, in a “Stop 
the Presses” item, pointed out that 
Microsoft Windows 95 overwrites a 
PC’s Master Boot Record—the first 
Windows version to do so and a FAQ 
for dual-booters ever since. More 
help came in the form of an ad for 
the “System Commander” boot man- 
ager, which offered an easy solution 
for multi-OS systems, with the bonus 
feature of fixing boot sector virus 
infections. 


—DON MARTI 
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Building a 
Call Center 
with LTSP 
and Soft 
Phones 


Need to equip an office with terminals and phones, 
all on a small budget? With LTSP and KPhone, you 
can do it with only terminals, sound cards and 
headsets. BY MICHAEL GEORGE 


new customer approached us with a need to provi- 

sion the office. The customer was receptive to open- 

source software and was interested in using Linux. 

Being a nonprofit organization, the budget for the 
project was tight. 

We provisioned the new office with a server running soft- 
ware from the Linux Terminal Server Project (LTSP) to make 
the desktop economical from the start. We then installed an 
Asterisk server as a PBX for the call center. To make things 
easier for the staff, we wanted to have a working soft phone on 
their terminals with headsets for hands-free operation. 

This article discusses the installation and use of the LTSP 
build environment to build Qt and KPhone so the staff mem- 
bers could run KPhone locally on their terminals. I do not dis- 
cuss the installation of Linux or Asterisk here, but I have 
included the relevant context for KPhone, which resides in the 
Asterisk sip.conf file. We used Gentoo for this particular LTSP 
server, but any Linux distribution can do the job. 


Software Needed 

The main software packages needed for this project were 
LTSP, KPhone and the LTSP build environment (LBE). LTSP 
easily provides thin-client access to a main server. We often 
recommend LTSP as an economical way to equip an office, 
because it focuses monetary resources on the main server 
rather than on the individual stations. The incremental cost of 
adding a new user to the office is relatively small, and adminis- 
tration is simplified. 

The customer’s new office is intended to be a small call 
center, so hands-free phone operation is a big benefit. We 
wanted to try using headsets and amplifiers that use a computer 
sound card for their connectivity rather than hardware phones. 
These headsets, coupled with software SIP phones on each 


user’s local station, allowed us to meet their phone needs with- 
out having to buy separate phone equipment. 

Because we already were using Asterisk (see the on-line 
Resources) as the PBX for the office, it seemed logical to 
use an open-source software phone. We decided to use 
KPhone (see Resources) as the software SIP phone, because 
it had proven reliable on standalone systems previously test- 
ed. One of the drawbacks of every SIP soft-phone package 
we investigated at the time was none supported a network- 
enabled sound protocol. As a result, they were required to 
run locally on the station that physically has the sound card. 
As these stations are thin clients that boot from the main 
server, KPhone needs to be resident in the filesystem on each 
station. When a user runs KPhone from the desktop, which 
runs on the server, the KPhone process needs to start in the 
local terminal environment. 

KPhone is not a standard part of the LTSP package, so we 
needed to build it inside the local stations’ root filesystem that 
is NFS-mounted from the server at boot time. Building soft- 
ware for the terminals’ root filesystem requires LBE (see 
Resources). Building software in LBE also requires that all 
necessary libraries be present in the filesystem. One of the 
other benefits of KPhone is that the Qt library is the only 
library required beyond those already in LTSP. 

Installation and configuration of LTSP are detailed in the 
LTSP documentation (see Resources). One deviation from the 
standard install of LTSP is that the DHCP configuration file 
must reference the root filesystem that LBE builds rather than 
the root filesystem installed with the LTSP package (Listing 1). 


Listing 1. Our LTSP Section of dhcpd.conf 


# LTSP Path Options 
option root-path 
"192.168.42.254:/usr/local/src/1be/opt/1tsp/i386"; 


#LTSP boot image (relative to the TFTP root) 
filename 
"/pxe2/pxelinux.0"; 


Technically, we did not need the LTSP package because 
LBE includes the necessary boot image and root filesystem. 
However, if you are not already familiar with LTSP, I recom- 
mend you install that package first and get it operational. 
Deploying LTSP involves the configuration of other standard 
software included with almost all Linux distributions: DHCP 
for assigning IP addresses, boot images and root filesystem 
information for the stations; TFTP for client stations to retrieve 
their boot images; and NFS for thin clients to remote-mount 
their root filesystems and the /home filesystem for running 
remote applications. Installing LTSP provides demo configura- 
tions for all of these packages that make setup much easier for 
a novice. 

The main LTSP documentation describes well most of 
the preparation for running applications locally on the 
clients. Their installation and configuration also are covered 
on the LTSP 4.1 Web page. In addition to the software men- 
tioned above, you also need to configure SSH client and 
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NIS on the server. 


SSH is the means we used for starting the process on the 
remote client. Notice that the LTSP 4.1 documentation demon- 
strates the use of rsh for launching the applications. Although 
that would work, the required demons for rsh no longer are 
part of the LTSP package. SSH is now the norm for launching 
local applications. You can find information about preparing 
for SSH launching of local applications in the Local 
Applications section of the LBE documentation. 

NIS is needed because the thin clients need to authenticate 
users through SSH as they launch the applications. NIS config- 
uration is guided by the NIS HOWTO. One item that was not 
immediately obvious from the documentation was that NIS 
would complain that /etc/publickey was not present. Creating 
that file with touch /etc/publickey solved the problem. 

Once all the supporting software is in place, configuring 
LTSP to run local applications is easy: set LOCAL_APPS = Y 
in /etc/Its.conf within the LTSP root filesystem. This causes 
the clients to mount the /home directory from the server with 
NFS. Also, NIS is made active by /var/yp/nicknames, 
/etc/yp/conf being created on the clients, domainname being 
run with the value of the NIS_DOMAIN entry in the Its.conf 
file and ypbind being run. The sshd demon also is activated 
on the client. 

For SSH operations to be transparent to users, we need 
SSH keys created without expecting users to do it themselves. 
To accomplish that, we installed superadduser in Gentoo, 
which is reported to be adduser from Slackware (see 
Resources) and modified it to generate the SSH keys automati- 
cally for the user when the user is created (Listing 2). 


Listing 2. Additions to /usr/sbin/superadduser 


# su to the user and generate their SSH keys 

su - "$LOGIN" -c "ssh-keygen -q -t dsa -C '' -N '' -f "$HME"/.ssh/id_dsa" 

# 

# cp the new public key to the authorized_keys file 

cp "$HME"/.ssh/id_dsa.pub "$HME"/.ssh/authorized_keys 

chown "$LOGIN":"$( echo $GID | awk '{print $2}')" "SHME"/.ssh/authorized_keys 
# 

# update the NIS stuff 

(cd /var/yp; make > /dev/null) 


Aside from configuring local applications to run on the 
client terminals, we also need to make sure the sound cards are 
active when the thin clients boot. Normally, one would set 
SOUND = Y, SOUND DAEMON = <nasd or esd>, VOLUME = 
<default volume level> and possibly SMODULE_01 = <ISA 
configuration string>. However, doing so not only causes 
the sound driver to be loaded into the kernel, but it also starts 
the sound demon, which we do not want. We need the sound 
card to be available for KPhone when it starts on the terminal. 

What we do instead is set SOUND = N to keep the normal 
sound system from being activated and MODULE_01 = <kernel 
module for the PCI soundcard>, because LTSP does not 
have isapnp support, so audio needs a PCI audio device. We 
also set RCFILE_10 = "kphone" to run the initial configuration 
script to ready the system for KPhone by using the audio 


device. Then, we put the KPhone script (Listing 3) in /etc/re.d 
in the clients’ root filesystem to enable access to the 
/dev/sound/* files. -rwrwrw access is not the most secure, but 
because only one user is running processes on the terminal at a 
time, it works fine. Finally, we turn on the microphone and 
adjust the gain and volume levels. 


Listing 3. <LTSP root>/etc/rc.d/kphone 


#!/bin/bash 
echo Setting up the system for using kphone locally 


echo change the permissions on the audio files... 
/bin/chmod 666 /dev/sound/* 


echo Turn on the microphone, adjust gain and volume 
/bin/aumix-minimal -m R 


echo Turn gain and volumes up to maximum 
/bin/aumix-minimal -m 85 
/bin/aumix-minimal -p 100 
/bin/aumix-minimal -v 100 


Building Qt and KPhone 

Now that you have the LTSP environment configured and oper- 
ational, you can build the LBE. Getting LBE from CVS is as 
simple as: 


cvs -d :pserver:anonymous@cvs.1tsp.org:/usr/local/cvsroot checkout -s 


You then need to su to root—using sudo with the LBE 
doesn’t reliably work—and run ./build_all. You can take 
a break here, as the build of LTSP in LBE takes some time 
to complete. 

Once you have the new root filesystem for the terminals 
built, change your DHCP configuration to refer to that boot 
image and root filesystem, and restart your DHCP server. 
You probably want to move /etc/Its.conf from your old LTSP 
root filesystem to the new one. You also should move the 
system-wide SSH known-host keys—the ones you created as 
per the Local Applications section of the LBE document—to 
the new filesystem. 

Now we need to build the Qt libraries and then KPhone 
inside the clients’ root filesystem. The LTSP build environment 
(LBE) makes this much more manageable. Adding packages 
for building in the environment amounts to creating a 
package.def file in a directory named for the package. 

The package.def files describe how to get, verify the download, 
unpack, configure, build and install the package software. 
The build script in the Itsp-srce directory then does a chroot 
and executes the build process. 

Through trial and error and discussions on the LTSP IRC 
channel (see Resources), we were able to construct the required 
package.def files (see Resources for those files). Constructing 
the package.def file for building Qt, in Itsp-sre/qt under the 
LBE root, was a straightforward process. Each build exported 
the same variables to the build environment. Notice also that 
threading is turned on explicitly at the CONFIGURE stage. 
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KPhone builds much more easily if Qt has threading enabled, 
but it is not enabled by default in Qt. 

Building KPhone was a bit more complicated. The 
package.def file (see Resources) works well enough, but 
the x-includes configuration option does not seem to 
change the resulting Makefiles. This would cause compila- 
tion errors when building trayicon.cpp. Manually adding 
-I/usr/X11R6/include to CFLAGS in kphone/kphone/kphone/ 
Makefile (Listing 4) after the configuration stage seemed to 
fix the problem, however. The steps to build KPhone in 
LBE are then: 


ltsp-src# ./build --configure --only=kphone 
ltsp-src# vi kphone/kphone/kphone/Makefile 

(Add "-I/usr/X11R6/include" to CFLAGS) 
ltsp-src# ./build --only=kphone 


We also noticed that the icons were not being located prop- 
erly by KPhone at first. Making a link to ../../share/kphone in 
opt/ltsp/i386/usr/share from the LBE root—/usr/share from the 
clients’ root—allowed KPhone to find the icons correctly. 


Listing 4. Modified CFLAGS in kphone/kphone/kphone/Makefile 


CFLAGS=-I/usr/X11R6/include -I/usr/qt/3/include \ 
-Wall -03 -I. -I../gsm -I../ilbc -I../dissipate2 \ 
-D_REENTRANT=1 -DQT_THREAD_SUPPORT=1 \ 
-DHAVE_LIBX11=1 -DHAVE_LIBXEXT=1 -DHAVE_LIBXT=1 \ 
-DHAVE_LIBICE=1 -DHAVE_LIBSM=1 -DHAVE_LIBPNG=1 \ 
-DSTDC_HEADERS=1 -DHAVE_FCNTL_H=1 \ 
-DHAVE_SYS_IOCTL_H=1 -DHAVE_UNISTD_H=1 \ 
-DHAVE_SELECT=1 -DINCLUDE_STDLIB_H=1 $(MOREDEFS) 


To run KPhone, we put a script in /usr/bin on the terminal 
server called kphone (Listing 5). This script simply opens 
access to the X server, determines the terminal at which the 
user is sitting and starts the KPhone process on that terminal. 


Listing 5. KPhone Script on the Server 


#!/bin/bash 

xhost + > /dev/null 

HOST="echo $DISPLAY | awk -F: '{ print $1 }'° 
export HOST 

ssh ${HOST} env DISPLAY=:0.0 /bin/kphone 


To make things easier for the users, we created an entry in 
the KMenu for KPhone that they can select or move onto their 
docks if they wish. This entry is created by adding the file 
kphone.desktop (Listing 6) to /usr/kde/3.3/share/applications/kde 
on the terminal server. 

The user then can select the KPhone menu item and launch 
KPhone (Figure 1). The first time the application is run, the 
user has to select File—Identity to open the Identity dialog 
(Figure 2) and enter the connection information. The data to 
enter here must match that information for the SIP accounts on 
the VoIP server (Asterisk in our case). Because KPhone stores 
its configuration in the user’s home directory, it needs be con- 
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Listing 6. kphone.desktop 


[Desktop Entry] 

Comment= 

Exec=kphone 
GenericName=O0ffice Telephone 
Icon=/usr/kde/3.3/share/icons/Locolor/32x32/apps/kab.p 
ng 

Name=kphone 

Path= 

StartupNotify=true 
Terminal=0 

TerminalOptions= 
Type=Application 
Categories=Qt;KDE;O0ffice 
X-KDE-SubstituteUID=false 
X-KDE-Username= 


File Preferences Help 
sip:kphonel1@192.168.42, 


Figure 1. The user’s desktop environment runs on the LTSP server, but KPhone 


runs locally. 


Full Name: 


|kphone]| 


User Part of SIP URL: 


[kphone1 


Host Part of SIP URL: 


[192.168.42.3 


Outbound Proxy (optional): 


ee —________. 


Authentication Username (optional): 


ee ee _—__ 


q-value between 0.0-1.0 (optional): 


poe een a ere _____ 


¥ Auto Register 
Registration : Active 


Unregister | 
help... Cancel | 


Figure 2. When first running KPhone for a new user, enter the information for the 


Asterisk server. 


figured only the first time the user starts KPhone. Because 
/home is NFS-mounted from the server, the station where users 
log in is their phone, so the phone effectively follows them if 
they should change workstations. Once users have registered 
with the server, they can make calls from the call dialog and 
DTME panel (Figure 3). 

Initially we had KPhone running, but the response time for 
any action was horrible. Any time the user would perform an 
action that caused an SIP message to be sent—dial a number, 
press a phone button on an active call, answer or hang up the 
phone—it would take nearly a minute for the action to occur. 

We determined that this problem was occurring because 
of a DNS name resolution issue that was waiting to time- 
out. The solution was to put entries into /etc/hosts for each 
of the stations that would be running KPhone, install 
dnsmasq on the terminal server and have the terminals ref- 
erence the terminal server as their DNS server, configured 


Remote: [226 | | 


Status: Audio: Unattached 


Figure 3. The KPhone call dialog works like a hardware phone. 


in dhcp.conf. There are other, perhaps better, ways to solve 
this issue, but this solution took minimal time to configure 
and run, and it worked. Finding the source of the problem 
was the hard part. 
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Gotchas 

There have been a couple drawbacks to this system. 
Occasionally KPhone closes for no given reason, which can 
be quite annoying. We have not yet determined the cause of 
this problem, and we hoped that upgrading KPhone to 4.1 
might help. 

The KPhone package.def file contains the necessary lines 
for building KPhone 4.1.1. The change to the Makefile men- 
tioned above for 4.0.5 still applies as of 4.1.1. Our preliminary 
tests indicate, however, that 4.1.1 has the same problem of 
closing suddenly for an unknown reason. We have inquired 
with the maintainers of KPhone to see if they can help, but so 
far we do not know the cause of the problem. 

Another drawback is that when the phone rings, it rings 
through the headset and gives a visual alert on the screen. If 
users are not in front of their terminals with their headsets 
on, they will not know that their phones are ringing. Once 
the call center is in full operation, operators probably will 
spend most of their time at the terminals, so this may not be 
a problem. 


Conclusion 

We now have KPhone installed and able to be run from any 
terminal attached to the LTSP server. Adding another user is as 
simple as creating an account for them on the server, adding an 
SIP phone entry for them on the phone system and having 
them configure KPhone. The terminal server is the single point 
of maintenance for everyone’s desktops. Even though KPhone 
runs locally on each terminal, the LTSP build environment is 
the single point of maintenance for all of them. 

The cost for the system is concentrated in the terminal 
server and phone system. The incremental cost for each 
new user is the cost of a low-end terminal and a sound card 
headset. This expense is much more cost effective than 
putting a full workstation at each desktop along with a 
headset-capable hard phone. 


Acknowledgements 
Thanks to James McQuillan at the Linux Terminal Server 
Project for his excellent documentation (LTSP and LBS) 
and everyone on the LTSP IRC channels that helped me 
get KPhone running locally. Also thanks to Thorsten 
Kukuk for authoring “The Linux NIS(YP)/NYS/NIS+ 
HOWTO”. Thanks to Mark Spencer, Digium and everyone 
involved with the Asterisk Project who have made open- 
source telephony a reality, as well as the author(s) of 
KPhone. And thanks to Trolltech, the creators of the Qt 
application framework. 

Resources for this article: www.linuxjournal.com/article/ 
8460.8 


Michael earned his degrees in Computer Science 
, from Michigan Technological University and 

| Purdue University more years ago than he likes 
to admit. He now lives in rural Michigan with his 
wife and five children. He has been using Linux 
since 1994 and now works with Ideal Solution, finding new 
and creative ways to put open-source software, including 
LTSP and Asterisk, to work for clients. He can be reached at 
george@idealso.com. 


ASA 
2¥.\ COMPUTERS 


www.asacomputers.com 
1-800-REAL-PCS 


Hardware Systems For The 


Open Source Community-Since 1989 
(Linux, FreeBSD, NetBSD, OpenBSD, Solaris, MS, etc.) 


The AMD Opteron™ processors deliver high-performance, 
scalable server solutions for the most advanced applications. 
Run both 32- and 64-bit applications simultaneously 


AMD Opteron” Value Server— —_ Front 1/0 Dual AMD Opteron” 
$195 Cluster Node—$1,600 


*1U 14.3” Deep + 1U Dual AMD Opteron™ Capable 


+ AMD Opteron™ 240 Font I/O 

*512MB RAM Max 8GB + Single 240 AMD Opteron™ 
+ 40GB IDE HDD *1GB RAM Max RAM 16GB 

+ 2x 10/100/1000 NIC + 80GB HDD 


* Options: CD, FD or 2nd HD, RAID 


8 Hot Swap Bays in 2U AMD 
Opteron"—$1,950 

*1 of 2AMD Opteron™ 240 
*512MB RAM Max 16GB 


+ Dual PCI Expansion Slot 


No Frills AMD Opteron” 
Storage Server—S$8,450 

* 6TB+ IDE/SATA Storage in 5U 
+ Dual AMD Opteron™ 240 


+ 3x80GB IDE RAID #5 + 512MB RAM 

+ 2xGigE, CD+FD + 6TB IDE Storage 

* Options: SATA/SCSI, + Dual GigE, CD 
Redundant PS + Options: 


SATA HDD, 
— = DVD+RW 


Your Custom Appliance Solution 
Let us know your needs, we will get you a solution 


“Your Logo Here” 


Custom Server, Storage, Cluster, etc. Solutions 
Please contact us for all type of SCSI to SCSI, Fibre to SATA, 
SAN Storage Solutions and other hardware needs. 


2354 Calle Del Mundo, Santa Clara, CA 95054 
Wwww.asacomputers.com 
Email: sales@asacomputers.com 
P: 1-800-REAL-PCS | FAX: 408-654-2910 


Prices and availability subject to change without notice. 
Not responsible for typographical errors. All brand names and logos 
are trademark of their respective companies. 


WWW.LINUXJOURNAL.COM OCTOBER 2005877 


INDEPTH 3-D SPATIAL AUDIO 


Dirt-Cheap 
3-D Spatial 
Audio 


With the addition of free audio software, an ordinary 
inexpensive surround sound card becomes the basis 
for a 3-D cube for simulation, visualization or gaming. 
BY ERIC KLEIN, GREG S. SCHMIDT, 

ERIK B. TOMLIN AND DENNIS G. BROWN 


any computer systems set up for advanced gam- 

ing include Dolby Surround Sound. The typical 

speaker configurations are 4.1—four speakers and 

one subwoofer— 5.1 and 7.1. This system is 
designed for all speakers to be located on a plane centered at 
the listener, and thus it is not possible to have a sound truly be 
emitted from above or below the listener, although some sys- 
tems attempt to simulate that effect. Imagine a game scenario 
where a monster is climbing down a wall above and behind the 
player while, at the same time, a mouse is scrambling across 
the floor behind the listener. In a planar surround system, the 
sound effects for both the monster and the mouse would come 
from the rear speakers, making it hard to distinguish the actual 
locations of the sound sources. 

With true 3-D spatial audio, the monster’s sound effects 
could be played from speakers located to the back upper-left 
and the mouse’s sound from speakers located to the back 
lower-left and back lower-right. In this setup, the player has a 
much better feel for what is creating the sound and where the 
sound is coming from. Now the player can arm the rocket 
launcher and turn toward the back upper-left directly and blast 
the monster—no need to aim toward the harmless mouse. 

Spatial sound has been available for several years and pri- 
marily is employed in immersive virtual environments. The 
systems are not in mass-scale production and often must be 
installed by professionals, making them costly and out of the 
reach of most home users. We have devised a low-cost true 3- 
D spatial audio solution that requires only inexpensive con- 
sumer-level hardware and open-source software. This solution 
allows for the arbitrary placement of speakers, not necessarily 
co-planar as in other systems. Our 3-D spatial audio solution is 
the first that we are aware of that provides true 3-D sound at 
such a low cost. 


Background on 3-D Spatial Audio 
Preliminary technology for 3-D spatial audio, Fantasound, first 


was developed in the late 1930s by Disney for the movie 
industry. Over the years, a great deal of work has been done to 
advance the field, especially by Dolby Laboratories. In the last 
few decades, researchers enabled personal computers to emit 
spatial audio. Today, spatial audio is commonplace in modern 
computer games. Home systems typically use headphones or a 
planar array of speakers, usually in a preset configuration, such 
as Dolby Surround Sound 5.1. 

Headphones present a unique opportunity to provide 
inexpensive 3-D audio. Algorithms that use head-related 
transfer functions (HRTFs) can create convincing 3-D spatial 
audio on headphones using a simple stereo sound card. 
HRTFs use data about how sound is transformed by the 
user’s body, especially the shape of the ears, for mapping 
sounds with 3-D positional sources. The technique relies 
heavily on applying different time delays for each ear. 
Ultimately, we decided not to use headphones, because we 
needed a system that scaled easily to many users. It was far 
more practical and cost efficient to use speakers. 

A number of high-cost professional-grade hardware pack- 
ages are available, such as the RME Hammerfall series, M- 
Audio Delta series and Lake Audio, that provide true 3-D spa- 
tial audio. Each package has a cost exceeding $1,000 US, 
boasts high sound quality and has a large array of features 
aimed at the professional market. Although the acoustic quality 
of these packages undoubtedly is higher than that of our low- 
cost 3-D audio solution in terms of audio clarity and fidelity, 
both options provide true 3-D spatial audio. 

When we started putting together a spatial audio system, no 
inexpensive hardware and software combination existed to pro- 
duce true 3-D spatial audio. Although there are software APIs 
that allow arbitrary, not necessarily co-planar positioning of 
sound sources, such as Microsoft DirectSound and the 
Advanced Linux Sound Architecture (ALSA), the low-level 
drivers officially support only the co-planar 4.1, 5.1 and 7.1 
speaker positions mentioned earlier. There is no way to tell the 
drivers that the speakers have been moved to an alternate con- 
figuration, for example, with speakers above or below the lis- 
tener. So even though software developers could position a 
sound above or below the user’s head, the low-level drivers 
still assumed the sound was emitted in a circle around the 
user’s head. The bulk of true 3-D spatial audio support comes 
from customized APIs. 

Tommi IImonen at the Helsinki University of Technology 
(HUT) developed a 3-D spatial audio API called Mustajuuri 
that is built on the ALSA drivers. The Mustajuuri API imple- 
ments Vector Base Amplitude Panning (VBAP), introduced by 
Ville Pulkki (see the on-line Resources), as the underlying 3-D 
spatial audio model. In short, VBAP is the algorithm responsi- 
ble for moving a sound across a 3-D array of speakers and 
making the sound appear to come from a specific direction. 
VBAP selects the three speakers closest to the virtual sound 
position and calculates the required volumes for each speaker. 
See Figure | for an example of how VBAP works. Mustajuuri 
also simulates depth for audio by using time delays and dis- 
tance attenuation. This makes it possible to position a sound 
anywhere in space relative to the listener. Mustajuuri already 
has been used to produce 3-D spatial audio using high-end 
audio cards, but up until now it has not supported low-end 
audio cards. 
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Figure 1. View of 3-D spatial audio test case in the immersive room. Visual depic- 
tions show from which speakers the sound is coming for the current view. For 
each sound, the three speakers closest to the virtual sound source are used to 


play the sound. Their volumes are varied based on the distance from the speaker 
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Figure 2. Audio Hardware Setup 


and a number of other factors. 


of this writing for as low as $90 US. Although it is possible 


Hardware Selection and Setup 


The hardware needed to set up a low-cost 3-D spatial audio 
system includes a commodity sound card with certain features, 


speakers and audio cables. Here, we 
describe our choices for hardware com- 
ponents and the steps needed to set up 
the hardware. Throughout the discus- 
sion, refer to Figure 2 for an illustration 
of the hardware interconnections, 
speaker placement and wiring. 

The first thing to consider is the 
number of speakers needed to produce 
3-D spatial audio for a specific applica- 
tion. A minimally encompassing setup 
produces sound from all directions 
around the user—left-right, front-back 
and up-down. The speakers can be 
placed in any configuration, but setting 
up the 3-D audio panning functions is 
not as simple for irregular configura- 
tions. We decided to use eight speakers 
in a cubic configuration, with each 
speaker at a vertex of the cube, as 
shown in Figure 2. There is nothing 
special about the speakers needed for 
this task—the choice is a matter of 
budget and taste. We used eight ampli- 
fied commercial-grade speakers for the 
simple reason that we already had them 
in our lab. 

The eight speakers require a sound 
card that can produce eight channels 
of audio. Of the low-cost commodity 
audio cards, the only applicable can- 
didates are the 7.1 cards. We chose 
the Creative Labs Audigy 2 card, 
which we found available at the time 


to produce eight independent channels of audio using more 
expensive sound cards, the Audigy 2 card is the only com- 
modity card we are aware of that has drivers in place to sup- 
port what we are doing. 
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It is important to understand the output from the card. In a 
typical Dolby Surround Sound 7.1 speaker arrangement, there 
are two front speakers, left and right; two side speakers; two 
rear speakers; a center speaker, which sits in front and above 
the video screen; and a subwoofer channel, the .1 speaker. The 
Audigy 2 ZS has three analog output jacks, an 1/8-inch mini- 
phone, labeled 1, 2 and 3, that provides line-level outputs for 
the eight speakers. Jack | is three-pole, meaning it carries three 
signals—two signals drive the front left and right speakers and 
the third is ground. Jacks 2 and 3 are four-pole, each carrying 
four signals. Jack 2 drives the rear speaker pair and a side 
speaker, while Jack 3 drives the subwoofer, the center speaker 
and the remaining side speaker. One final consideration is these 
signals are unamplified line-level, so the speakers need to be 
the amplified type that accept line-level inputs. Alternatively, a 
separate amplifier or set of amplifiers should be used between 
the sound card and the speakers. 

The next step is to install the speakers. Many speakers 
designed for surround use include mounts, but speaker mounts 
are available commercially for a number of other speaker 
types. In our application, we already had the cubic infrastruc- 
ture in place and used custom mounts to attach the speakers to 
the cube. The simpler the speaker configuration, the simpler 
the software configuration can be—that process is explained 
later in this article. 

Finally, the speakers must be connected to the audio card. 
How one connects speakers to the Audigy 2 depends on the 
type of speaker and amplification used. A trip to a favorite 
electronics store should yield any necessary connectors. In our 
case, the speakers each have a two-pole 1/4-inch phone jack, 
so we needed to split the three combined outputs of the sound 
card into eight separate signals. For Jack 1, we used a readily 
available 1/8-inch-stereo-to-dual-RCA adapter. For Jacks 2 and 
3, we found similar adapters with four poles and three RCA 
connectors. These adapters are used most commonly with cam- 
corders, when the three signals are used for composite video 
and stereo audio. These adapters gave us eight separate RCA 
connectors, and after obtaining eight long RCA-to-1/4-inch- 
mono cables, we were set. 

In our final configuration, we used an Alesis Studio 32 
mixer. This device fits in-line between the audio card’s outputs 
and speakers’ inputs and allows fine-tuning of the volume lev- 
els. Although the mixer made it a little easier to test and tune 
the audio, it wasn’t truly necessary, as the same adjustments 
can be made in software. 


Software Selection and Setup 

The software solution for low-cost 3-D spatial audio is best 
described by the layered hierarchy shown in Figure 3. The soft- 
ware layers required to interface with the sound cards include 
low-level audio drivers and a 3-D spatial audio API. We 
focused our primary development efforts on Linux because of 
easy access to the source code for low-level audio drivers and 
the overall support community that exists for developers work- 
ing on projects such as ours. 

For the driver layer, we chose ALSA, which was men- 
tioned previously. ALSA provides audio and musical instru- 
ment digital interface (MIDI) functionality to the Linux 
operating system. It supports many types of audio hardware, 
ranging from consumer sound cards to professional multi- 


channel audio interfaces. 

We selected ALSA because it appeared to require the least 
effort to generate the eight channels we needed for 3-D spatial 
audio. Until we modified the ALSA driver to access all eight 
channels, it supported only six channels (5.1) on the Audigy 2. 
These changes have been incorporated into ALSA, but they 
may or may not be in a release version at publication time. In 
that case, one can get the latest source and build it—be sure to 
include the emu10k1 sound card argument when using the 


High teve! +SOftware Application 
&. 3D Spatial Audio API 
a Audio Device Drivers 
<= 

towlevel Operating System 


Figure 3. Audio Driver Layers 


/configure script so that the ALSA driver recognizes the 
Audigy card. 

After the driver is set up, the 3-D spatial sound API can 
be installed. It distributes sound effects from a given 3-D 
position to the appropriate audio channels. Although there 
are quite a few APIs to choose from, we chose Mustajuuri, 
as mentioned previously. The Mustajuuri software works 
with ALSA and provides 3-D panning over an arbitrary array 
of speakers using the VBAP algorithm, also described previ- 
ously. The Mustajuuri API provides all of the features need- 
ed for a basic 3-D positional sound system and is fairly easy 
to extend. Over the course of this project, we made several 
minor source code modifications, and they are included in 
the October 2004 release. 

Mustajuuri does its magic by way of a module called the 
Mixer, which mixes multiple sound sources—sound files, 
microphone inputs or other sources—into individual audio 
streams. These streams then are piped into a panning module, 
which is responsible for routing each input signal to the appro- 
priate speakers, setting the correct gain and time delay at each 
speaker and mixing multiple streams meant for the same 
speaker into a single stream to be sent to that speaker. It does 
the routing and gain calculations based on VBAP, and some 
additional gain and delay calculations are based on distance. 
The result is each incoming sound source to the panning mod- 
ule leaves from a set of three speakers, and the resulting sound 
appears to come from a specific 3-D position in space. Doppler 
shifting also is simulated. 

Once Mustajuuri is compiled and installed, several tasks 
must be performed to configure the software to work with the 
given 3-D speaker array. 


Configuring ALSA 
ALSA needs to know how to communicate with all eight chan- 
nels of the audio card. This normally would be achieved sim- 
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ply by using the device named surround71, but it is not fully 
compatible with the spatial sound API Mustajuuri. Mustajuuri 
requires support for input channels. The device surround71 
supports eight output channels but no input channels. a 7 ee ie 
Therefore, it is necessary to define a new device that has eight wwwa.linuxjournal.com 
output channels and some input channels. 
Pp d some inpt : ADVERTISING SERVICES REGIONAL ADVERTISING SALES 
In order to meet this requirement, an asymmetric device VP OF SALES AND MARKETING NORTHERN USA 
is defined. The device is called asymmetric because the num- Carlie Fairchild, carlie@ssc.com oo ee 
ber of input and output channels are not necessarily the +1 206-782-7733 X110, 866-423-7722 FAX 
same. Notice that the number of input channels is not stated ween eS eae cba 6 
erase is é Laura Whiteman, laura@ssc.com 
explicitly. ALSA determines the number of input channels FOR GENERAL AD INQUIRIES 306-782-7733 x 119 
automatically and assigns the maximum; the Audigy card we e-mail ads@ssc.com EASTERN USA 
used has two. or see wwwilinuxjournal.com/advertising a ea 
To configure ALSA, add the following text to the file please ited inkamatonal advertising +1 905-947-8849 FAX 
/etc/asound.conf or create the file if necessary. This file holds inquiries to VP of Sales and Marketing, INTERNATIONAL 
. ‘ 7 ‘ Carlie Fairchild Annie Tiemann, annie@ssc.com 
information about user-defined devices, so we use the follow- : 866-965-6646 (toll-free) 
ing text to add an asymmetric device called eightout: Aaverucee Pogew | advertiser Pages 
E ABERDEEN, LLC 9 Layer 42 NeETWorRKS 43 
ctl.ej ghtout { www.aberdeeninc.com www.layer42.net 
type hw 
se 0 APPRO HPC So tutions 19 LEVANTA 13 
car appro.com wwwlevanta.com 
} ASA COMPUTERS 75,77 Linux JOURNAL 91 
www.asacomputers.com www linuxjournal.com 
pcm. ei ghtout { AVOCENT 53 Linux Wortp Expo UK 33 
type asym www.avocent.com www.turretgroup.cm 
playback.pcm { CARINET 67 LPI 65 
type route www.complexdrive.com wwwlpi.org 
slave.pcm surround/71 ClARA TECHNOLOGY 16, 17 MBX Z 
www.ciara-tech.com www.mbx.com 
ttable.0.0 1 
ttable.1.1 1 Coralp, INc. 21 Microway, INc. C4, 83 
bl 2 2 1 www.coraid.com www.microway.com 
ttable.2. 
Coyote Point 61 Novett 47, 49,51 
ttable.3.3 1 www.coyotepoint.com www.novell.com 
ttable.4.4 1 
CycLapes CORPORATION c2,1,11 Open Source PROFESSIONAL INSTITUTE 69 
ttable.5.5 1 www.cyclades.com www.ospinstitute.com 
ttable.6.6 1 Emac Inc 44 PENGUIN COMPUTING 23 
ttable.7.7 1 www.emacinc.com Wwww.penguincomputing.com 
} EMPERORLINUX 15 PFU Systems 57 
capture.pcm { www.emperorlinux.com www.pfusystems.com 
ty pe hw Etnus 85 R CuBeD TECHNOLOGIES 25 
card 0 www.etnus.com www.rcubedtech.com 
} FairCom CorPoraTiON 39 Rackspace MANAGED HostiNG 3: 
} www.faircom.com www.rackspace.com 
FREE SOFTWARE FOUNDATION 93 RoBoNexus 63 
: : www.gnupress.org www.robonexus.com 
Next, an environment variable must be set to allow 
Mustajuuri to talk to the audio card through ALSA. Set the fol- bias 32) | See 2 
‘ i; 2 www.google.com/|j www.sbei.com 
lowing environment variable: 
HurrICANE ELECTRIC 55 S¢c|o5 89 
www.he.net sc05.supercomputing.org 
export MJ AUDIO _CONF= \ 
~ . IMAGESTREAM INTERNET SOLUTIONS 37 Servers DiReEcT 41 
"TInput=2=hw:0,0 | Output=8=eightout" www.imagestrearn.com www.serversdirect.com 
ao 2 2 INTEGRATED IT SoLuTIONS DBA SAG ELectronics = 87 STORIX SOFTWARE 59 
Once this is done, Mustajuuri should be able to output www.sagelectronics.com wwwstorix.com 
audio through all eight channels of the audio card. nt 29 | sucaeCRM, Inc. 3 
intel.com/go/xeon www.sugarcrm.com 
Configuring the Mustajuuri Mixer Panel INTEL PREMIER PROVIDER 28 TECHNOLOGIC SYSTEMS 45 
Mustajuuri uses a mixer-board-style GUI for sending input Wn Tee spore A a 
audio streams to a speaker array, combining them or just pass- IRON Systems 79 | TUX macazne 90 
. . . . www.ironsystems.com www.tuxmagazine.com 
ing them through intact. The input streams can come either ° 
from sound files or from live sources, such as a microphone. aE es CORe Zr |THE PORTAND Grade 30, 34 
- www.iwillusa.com Www.pgroup.com 
The GUI lays out several strips of channels that can be 
: I = F 7 A JTL Networks 92 ZT Group INTERNATIONAL = 27 
assigned different functions applied in a sequential process. worwitlnet/ swwateroupconh 
WWW.LINUXJOURNAL.COM OCTOBER 2005881 


INDEPTH 3-D SPATIAL AUDIO 


Some example functions are 
input, send (to speaker), 
amplitude gain, panning and 
synthesizer. The gain and 
panning modify how the 
audio is distributed to indi- 
vidual output audio channels. 

The Mixer Panel configu- 
ration we use is shown in 
Figure 4, which uses two 
mixer strips. The first has 
two interesting channels: a 
synthesizer channel, which 
manages the sound files, and 
a panning module, which 
handles the VBAP-based 
panning across speakers. The 
second strip is used to man- 
age remote connections from 
external applications and 
does not accept an audio 
stream as input. It sends 
commands to the synthesizer 
and the VBAP module. 

To create a similar con- 
figuration, launch Mustajuuri 
and create a new mixer from 
the File menu. This mixer 
has several strips already, 
and all of these strips essen- 
tially are blank. The number 
of strips and the number of 
modules per strip can be 
changed using the Edit menu, 
if needed. Modules can be 
assigned by clicking with the mouse on a particular slot. To 
adjust the module’s properties, simply click on the blue link 
defining the module’s type, such as Synth! or Mixer Input. The 
Strip X button at the top of a strip can be used to modify and 
remove the modules in any slot in that strip. All mixer configu- 
ration changes are saved by using the save options from the 
File menu. The resulting configuration file, for example, 
SpatialAudio.mj, is specified on the command line when 
Mustajuuri is called. 


i 
ie i 


Panorate with 
Setup s.vbap 


| 


re 
qi 


i 


Figure 4. Screenshot of the 
Mustajuuri Mixer GUI (Tommi 


IIlmonen, Mustajuuri-2004) 


Specifying Speaker Placement 

In order to use VBAP, it is necessary for Mustajuuri to know 
the locations of the speakers in the 3-D array. Mustajuuri does 
this through a configuration file that is specified as part of 
the VBAP module setup; this module was created as part 

of configuring the Mustajuuri Mixer Panel. This file specifies 
the azimuth and elevation angles (in degrees) for each speaker 
relative to the listener. Because our system uses eight speakers 
arranged in a cube configuration, our configuration file is 
specified as follows: 


3 # dimensionality 

# Azimuth, followed by elevation. 
# @ @ would be straight ahead. 
-45 45 # Front upper left 


45 45 Front upper right 
-135 45 Back upper left 
135 45 Back upper right 


45 -45 
-135 -45 
135 -45 


# 

# 

# 
-45 -45 # Front lower left 

# Front lower right 

# Back lower left 

# Back lower right 

This configuration file is used by the main configuration 

file for the Mustajuuri Mixer. This file assumes that all speak- 
ers are equidistant from the listener. If this is not the case, 
adjust the gain and delay for each speaker manually by using 
the Mustajuuri Mixer. In our system, such adjustments were 
not necessary, and it would involve significant work if it were 
necessary. The easiest solution is to try to place all speakers 
equidistant from the ideal listening position. 


Configuring the Sound File Loader 

In order to use a sound file from Mustajuuri, that sound file 
must be known to Mustajuuri at load time. The mechanism 
used to do that is a configuration file that specifies all of the 
sound files that possibly might be used by Mustajuuri. This 
configuration file is used by the synthesizer mixer module, 
which was created when we configured the Mustajuuri Mixer 
Panel above. A sample configuration file that loads three sound 
files follows. Once a file is created, any name can be assigned; 
make sure the synthesizer module points to that file: 


unusevoices *-stk 
polyphony 48 


sample audioeffectl.wav 
sample audioeffect2.wav 
sample sudioeffect3.wav 


The polyphony line specifies the maximum number of 
audio files that should be loaded by Mustajuuri, so it should be 
at least as large as the number of audio files listed in this file. 
The last three lines specify three sample audio WAV files. Any 
audio files specified here must be placed in a directory speci- 
fied separately as part of the synthesizer module configuration. 
The unusevoices line is a somewhat more-advanced setting, but 
one that should help improve efficiency somewhat. 


Configuring Mustajuuri for Remote Control 
Mustajuuri is designed to act as a standalone program to 
manipulate audio, not as a library to be linked against by 
another application. To control Mustajuuri from another appli- 
cation, as is the case with our project, two steps are required. 
The first involves setting up Mustajuuri to listen for control 
commands over the network. The second step consists of writ- 
ing a simple API in the main application to talk to Mustajuuri. 
To get Mustajuuri to accept commands from the network, a 
network module must be loaded. This network module is the 
only way for an external application to control Mustajuuri, 
even if both the application and Mustajuuri are running on the 
same machine. Adding this module is a simple task and simply 
requires configuring which port Mustajuuri will listen to; the 
default port is 10030. This module automatically communi- 
cates with the synthesizer module and the VBAP modules, if 
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they are in the mixer. 

In order to control Mustajuuri from an application, it is nec- 
essary to add code included with the Mustajuuri API into the 
application. We present example code segments here that show 
how to connect to a remote audio server, play audio specified 
coming from a given 3-D position and change the position of a 
sound source and listener position. 

The first code segment shows the initial commands to con- 
nect the application to the remote Mustajuuri audio server and 
initialize it. If Mustajuuri is running on another machine, 
change the address to reflect this. To change the default port, 
10030, that Mustajuuri listens to, specify the new port in the 
address string, for example, mjserver.mydomain.com: 12345. 
The two objects that we created, one instance each of 
AC_Control and AC_VrControl, are used later to send com- 
mands to Mustajuuri: 


// connecting to a remote server 
#include <ac_vr_control.h> 


AC_Control acControl = 

new AC_Control(); 
char* mjServerAddress = "127.0.0.1"; 
if(!acControl->init(mjServerAddress) ) 
{ 


// error handling code here 


AC_VrControl acVrControl = 
new AC_VrControl( acControl ); 


The next code segment shows how to specify the position 
of the source of the audio and play it. One interesting thing to 
note here is that the variable outputChannel identifies the 
intended sound source to use. The number of supported sound 
sources was specified in the synthesizer module from the 
Configuring the Mustajuuri Mixer Panel section, and 
outputChannel should be between 0 and the number of sources, 
minus one. The variable soundFilename should not have a path 
as part of the filename. The filename should be one of the files 
listed in the configuration file created as part of configuring the 
Sound File Loader file. Lastly, the soundLevel is essentially 
the initial gain level for the new sound. This needs to be exper- 
imented with to find an appropriate setting: 


AC_Vector3 location( 
positionx, 
positiony, 
positionZ) ; 


int soundId = 
acVrControl->playSample( 

outputChannel, 
soundFilename, 
soundLevel, 
location, 
true, 
Oy .O)5 


The last code segment shows how to reposition the sound 


source location and the orientation and position of the listener. 
The outputChannel variable refers to the sound source that is 
desired to be moved and should be the same value used to call 
playSample from the previous example. The listenerRotation 
matrix specifies the orientation of the listener relative to the 
world, and the worldRotation matrix specifies the orientation 
of the world relative to the speakers: 


// reposition a source of sound 
AC_Vector3 location( 
positionx, 
positiony, 
positionZ) ; 


acVrControl->moveSource(outputChannel, 
@.05, location); 


// reposition the listener orientation 
AC_Matrix3 listenerRotation( 
listener rotation matrix ... ); 
AC_Matrix3 worldRotation( 
. world rotation matrix ... ); 


acVrControl->setTransformations ( 
location, listenerRotation, 
worldRotation, 0.05); 


Hardware Testing and Calibration 

We tested the hardware design of our 3-D spatial audio system 
by integrating the hardware with our four-wall immersive vir- 
tual reality room at the Virtual Reality Laboratory, part of the 
Naval Research Laboratory in Washington, DC. We arranged 
the speakers in a cube array and placed them at the corners of 
the immersive room, as shown in Figure 5. We designated a 
1.2GHz Red Hat Linux machine as an audio server and 
installed an Audigy 2 ZS card. We connected the speakers 
using the cabling described before and tried the system both 
with and without the mixing board mentioned earlier. 

While Mustajuuri was running with three audio sources in 
motion, CPU utilization on this machine generally was less 
than 20%, and the memory usage was negligible. Further sav- 
ings could be realized, of course, by using optimized compiler 
settings rather than the debug settings we used. 


4-Wall Immersive 
Room 


Figure 5. Immersive room depiction showing placement of speakers in a cube 


array and audio coming from the user’s front upper-right direction; the lines are 
colored red. 
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We tested the outputs from each speaker to determine the 
range of intensities that could be played on each channel. We 
listened to each speaker individually for sound quality, sound 
balance and percussive resonance. The easiest aspect to listen 
for is the sound balance between treble and bass. If one of the 
two obviously is higher than the other, adjust the related fre- 
quency filters as needed. For example, if there is too much 
bass, decrease the bass and/or increase the treble. If there 
seems to be excessive low-end or high-end noise, adding a 
low-pass or high-pass filter may be necessary. 

Another easy aspect to listen for is speaker distortion. 
Simply put, if the speakers are so loud that the sound produced 
is bad, lower the volume of the speaker. If a given set of speak- 
ers cannot produce quality sound at an 


made in software with the ALSA drivers and Mustajuuri if a 
mixer is not available. 


Software Testing 

We tested the software by integrating sound into an existing in- 
house simulation platform, BARS-Utopia, that operates on a 
Linux visualization cluster from ORAD Incorporated, which 
drives our immersive room. BARS-Utopia supports several vir- 
tual world databases, interaction methods and spatial audio. 
However, no support was available for interacting with the 
Mustajuuri API in particular, so we implemented a plugin to 
bridge the BARS-Utopia spatial audio support with Mustajuuri. 
BARS-Utopia already contains all of the information needed 


acceptable volume, it may be necessary 
to acquire more powerful speakers. 
One of the hardest aspects to listen 
for is the resonance of percussive 
sounds generated by a speaker. This 
quality basically is how much the 
sound echoes from where the speaker 
is located. Adjustments have to be 
made if it sounds like a speaker is 
reverberating with percussive sounds. 
Depending on the quality of the 
speaker and the quality of the mixer 
board, this problem may be corrected 
to some degree by continuing to filter 
the signal. For excessively bad cases, 
hard objects such as exposed metal, 
concrete, hard plastic and even glass 
should be covered with a sound damp- 
ening material such as cloth or foam. 
Once each speaker is calibrated, 
the entire setup has to be balanced. 
This can be done either by using 
devices designed to measure acoustic 
levels or by listening to the speaker 
from the predetermined center of the 
3-D speaker array. Either way, the 
gain of each speaker should be 
adjusted until the same audio intensi- 
ty level is received from each chan- 
nel. Keep in mind that the outputs for 
each channel of the audio card were 
customized by the manufacturer for 
the intensity requirements for each 
type of speaker—satellite, center and 
subwoofer—normally attached in a 
surround configuration, and the 
intensity output for each type differs. 
There are many published methods 
for dealing with this problem, but we 
went with the low-tech solution of 
having someone stand and listen at 
the center of our speaker array. We 
set the software control to maximum 
gain and adjusted the mixer board 
based on feedback from the listener. 
Remember that these changes can be 
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INDEPTH 3-D SPATIAL AUDIO 


Figure 6. Shows a user interacting with a scene with multiple sound sources—a 


tank, helicopter and a jet that is not visible. The nearest speakers are determined 


for each sound source and the output is mixed at each speaker, if overlapping. 


by Mustajuuri, such as sound source positions, listener position 
and orientation and sound source creating/deletion notifica- 
tions—the plugin simply translates that data into a form that 
Mustajuuri understands. 

When the plugin was completed, we tested and debugged 
the new system. The primary software adjustments we made 
were to the attenuation level of the audio channel outputs. 
Mustajuuri uses a simple attenuation model and requires some 
manual tweaking for the expected environment, things such as 
outdoor, indoor, time of year and so on. In the real world, 
sound attenuation rates are quite complicated and are influ- 
enced by factors such as temperature, humidity and the fre- 
quency makeup of the sound. 

We tested the sound system by implementing several sce- 
narios, each with a different scene dataset and different audio 
effects attached to an animated object. Before the audio objects 
were animated, we evaluated several volume levels and several 
distances away for each object. Figure 1 shows a simple sce- 
nario we designed and tested—the sound effects of a car. When 
we finished testing the volume and distance effects, we gener- 
ated an animated path for the car to follow. 

Figure 6 shows a more complex scenario with three audio 
sources, tank, jet and helicopter—the jet is off the screen and 
not shown in view. We performed some simple tests to see how 
many sound sources interacted together. It was of primary 
importance that the jet, typically far away, not sound too quiet, 
while the tank and helicopter, typically closer to the camera, 
not dominate the aural bandwidth. As a result, some minor 
tweaking was done on both the far and near objects’ attenua- 
tion parameters. 


System Validation 

After all of the testing and calibration was completed, we per- 
formed two informal, qualitative user tests that would help us 
validate our new low-cost spatial audio system. The first test 
evaluated how the new sound system configuration with eight 
speakers compared with our previous planar configuration 
containing four speakers. The prior configuration simply used 


the four speakers on the top of the cube array. We realize that 
directly comparing these two configurations is somewhat 
biased, due to the placement of the four-speaker array being 
located above the user’s head. It would be more fair to com- 
pare against a four-speaker array located at the height of the 
user. However, by using the top four speakers, we were able to 
switch between the two configurations without dismantling 
our installation. 

We performed the experiment by asking a few test subjects 
to stand in the middle of the immersive room and listen to 
sounds played for each configuration. We played different 
sequences of audio on both speaker configurations and made 
use of the full range of speakers available. The subjects were 
not told which configurations were being used, nor in which 
order the pairs of configurations were presented. Several itera- 
tions of the pairs of configurations were tried for each subject. 
After each pair was presented, the subjects rated the two sys- 
tems. Admittedly, this was not a scientific test, as is evidenced 
by several unaddressed biases, but all test subjects clearly pre- 
ferred the eight-speaker configuration. 

The second user test evaluated how well the listener is 
able to localize the source of the audio using the eight- 
speaker configuration. Again, the subjects were tested and 
each was asked to stand in the center of the immersive 
room. Each subject was presented with several sounds 
played one at a time and originating from different positions 
surrounding the subject. The subjects were asked to point in 
the direction of the sound source, as they heard it. The visu- 
al system was not running, so the users did not get visual 
cues as to the sound source’s location. The subjects were 
able to localize the sounds with a high degree of accuracy, 
especially with respect to elevation. 

The implementation of our 3-D spatial audio system 
integrated with our immersive room really enhanced the 
simulation and training demos we have. Our completed sys- 
tem has improved dramatically the sense of immersion when 
running the demos. A simulation user easily perceives heli- 
copters and jets flying overhead and a tank rumbling down 
one of the many streets nearby in the virtual world. The per- 
ception of depth from the source of audio is conveyed accu- 
rately and also includes doppler effects. Our system is a step 
above a four-speaker solution when we had previously used 
the Microsoft DirectSound API. It also is a good replace- 
ment for the capable but outdated and unsupported eight- 
speaker solution we had running using another expensive 
hardware and software platform. 


Conclusions and Future Work 
We have devised a true 3-D spatial audio solution that is low 
cost and has comparable quality to expensive high-end com- 
mercial systems. The 3-D spatial audio solution allows sound 
effects to be generated from all directions surrounding a 
user, not only from planar directions. We accomplished this 
feat by using only commodity hardware and open-source 
software. We feel this feature, now available at an affordable 
price, creates numerous options for game and virtual reality 
system developers. 

We feel our system leads the way for others to devise simi- 
lar solutions with current and future commodity audio equip- 
ment. The developer needs only to purchase a Dolby Surround 
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Sound 7.1 audio card, four pairs of low- 
cost speakers and audio cables. We 
spent less than $150 US on hardware— 
Audigy 2 audio card and audio cables— 
as we already had speakers available. 
From start to finish, including hardware 
and software debugging, configuring 
and testing, we spent less than a month 
developing the low-cost 3-D spatial 
audio system. We feel that using this 
document as a guide, it should be possi- 
ble for others to implement this system 
in less than a week. 
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INDEPTH TAMING THE TODO 


Taming the 
TODO 


Buried under a mass of sticky notes? If you worry 
about forgetting important tasks or you want to 
schedule things efficiently, here are some ways to 
get organized. BY SACHA CHUA 


n this article, I offer some ways to manage your tasks. 

From simple text files to full-blown personal information 

managers (PIMs), there’s bound to be one method that 

fits your way of working. I also share some tips on man- 
aging your tasks and tell you about how I fit a task manager to 
my way of working. 


You Want the Works 

If you’re accustomed to the advanced task management fea- 
tures of Microsoft Outlook and other proprietary PIMs, then 
Ximian Evolution and the KDE PIM suite are great fits for 
you. Ximian Evolution was developed for the GNOME user 
environment, and the KDE PIM suite is part of KDE, but each 
is usable with other desktop environments. 

Offering a polished interface for creating and managing 
tasks, attaching files and even synchronizing with personal dig- 
ital assistants (PDAs), these full-fledged personal information 
managers can help you tame your to-do lists (TODOs) in style. 


You Want to Keep Things Simple 

Sometimes the simplest method is the best. Keep tasks in a 
plain-text file, and you’re already well on your way to taming 
your TODOs. Text files win in terms of flexibility. You can 
keep your lists in any format you want and edit them using 
your favorite editor. You also can share them with others 
through e-mail or the World Wide Web. You even can keep 
them backed up and synchronized with other computers using 
tools such as rsync and CVS. 

Memorize keyboard shortcuts for copy and paste. 
Incremental search is a great way to jump to tasks if you 
remember a small part of the description. Your text editor then 
can display matches as you type in characters. Check out your 
text editor’s features for more help. 

Beyond the basics, a little bit of programming makes 
TODOs easier to keep. Write a small program or shell script to 
add items from the command line or a keyboard shortcut. The 
less effort it takes to write down a task, the more you’ ll 
remember, so automate as much as you can. You can sort tasks 
manually by copying and pasting lines in your TODO list or 


even writing programs to put everything together. 

For more software support, check out Freshmeat.net for 
hundreds of simple TODO managers. If you know how to pro- 
gram, pick a TODO manager in a language you know or would 
like to learn. Extending a manager’s capabilities not only helps 
you grow as a programmer but also lets you tailor it to your 
particular quirks. 


You Get Most of Your Tasks through E-mail 

E-mail is a popular way to keep track of tasks. If you practical- 
ly live in your e-mail client, why not use it to keep track of the 
things you need to do? You can forward messages or write 
yourself reminders. Use meaningful subjects to make it easier 
to get a bird’s-eye view of your messages. 

Watch out for information overload, however. You may 
need to find that urgent TODO in an archive of thousands of 
messages. Check out your mail client’s features for options on 
how to tag messages. Use folders or labels to flag messages for 
follow-up action. Tag or file messages as TODO, and remove 
the label or change it to “done” after you finish the task. 
Keeping track of tasks is easier with full-fledged PIMs, such as 
Evolution and KDE PIM, which allow you to mark a message 
for follow-up or convert it to a task. 

What about small tasks? It might seem silly to e-mail your- 
self a reminder to buy milk, but unless all of these TODOs are 
written down somewhere, you’re going to spend mental energy 
thinking about them. You therefore may need to supplement 
your inbox with a way to keep track of smaller tasks. 

If most of your tasks can be accomplished quickly and you 
can keep your inbox manageable, e-mail is a convenient way to 
keep track of your tasks. 


You Work with a Lot of People on Tasks 

Many software projects use request trackers to make sure 
that bug reports and feature requests don’t slip through the 
cracks. You can use one to keep track of your personal 
TODOs too. Although a request tracker requires a lot of set- 
up time and effort, you reap the benefits of a solid project 
management system. 

Request trackers, also known as bug-tracking systems 
(BTSes) or issue trackers, archive all of the messages related to 
a TODO, making them great for tasks occurring over long peri- 
ods of time and tasks when you need to collaborate with other 
people. You can send the e-mail address or URL for a task to 
other people so they can confirm your work or add comments. 

Request trackers can produce task-related graphs. For 
example, you can track the increase or decrease in open, 
resolved and closed tasks over time to get a rough estimate of 
when you’re most productive or overloaded. 

If most of your tasks require input from others, check out 
programs such as RequestTracker and Bugzilla. With a good 
bug-tracking system in place, you easily can keep track of what 
you’re waiting for and from whom. 


You Practically Live in Your Web Browser 

Web-based TODO lists are a fun and easy way to create task 
lists you can share with other people. If you always have a 
Web browser open or you need to keep non-techies updated, a 
Web-based TODO list might be a handy way to keep track of 
your tasks. New services such as Ta-da and Backpackit use 


88HOCTOBER 2005 WWW.LINUXJOURNAL.COM 


\, SC\05 Gateway to Discovery 


SC|05 Highlights: 


Keynote Address by Microsoft Chairman and Chief Software 
Architect Bill Gates 


SC|OS5 initiatives HPC Analytics and StorCloud will push the boundaries 
of high performance storage, analytics and networking applications 


Technical, Education and SC Global Programs offer attendees a gateway 
into innovative ideas and expertise in HPC, networking and storage 


A dynamic exhibition featuring more than 200 of the world's 
leading HPC industry and research organizations 


SUPER MPUTING 
j 4 [e =] a ° 
ie e€ = . a Ss 
SC } se = | } GC s Ss 
[ Pal =) 
Registration Date: August 2005 7 Sponsors: 
Register: http://sc05.supercomputing.org/ ISDE Ganges tecly 
See you in Seattle! . ~~ © * 
soa US ‘ oe 7 — . At 
4 4c! ) Saw 
> NS f ae = 
| I£&Ee 


INDEPTH TAMING THE TODO 


JavaScript and XHTML tricks to provide a great user interface. 

Look for a bookmarklet or extension that lets you easily 
create TODOs. Make your task overview the default page in 
your browser so that you’re sure to review them daily. 


You’re Always on the Move 
If you spend a lot of time on the move, you’ve probably 
thought about getting yourself a PDA. Both Evolution and 
KDE PIM can synchronize your tasks with Palm-based 
PDAs, making them ideal for the mobile warrior. Libraries 
such as coldsync can help you support synchronization for 
your custom hacks. 

My productivity tool of choice is a pack of 3"x5" index 


cards held together with a fold-back clip or rubber band. 
Affectionately called the “Hipster PDA” by productivity geeks, 
this surprisingly effective low-tech tool is a great way to keep 
track of tasks. 

Write down your tasks, one per index card. You can write 
down subtasks and notes as well. Shuffle through your tasks 
while waiting or sort them by the context you can perform the 
tasks in. Rip the card up after completing your TODO for an 
extremely satisfying end. 

Print useful data onto cards. Around 50 names and contact 
numbers can fit on an index card if you use a really small 
font. Month and year calendars also are handy. No hardware 
worries, no productivity-sapping games and no hassles make 

the Hipster PDA great for people on 
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the go. 


Tips for Taming your TODOs 

Got an idea about what to use to 
manage your tasks? Well, now here 
are some tips for keeping on top 
of everything. 


Make It as Easy as Possible 
“Hmm, that looks interesting”, you 
think. “Let’s try it out.” You switch to 
your task manager to write down that 
TODO. Oops, you still need to open 
the application. Now you have to 
arrange your windows so you can see 
the article. Wait, you need to copy the 
URL. By the time you have it all set 
up, you might’ ve forgotten what you 
wanted to write down in the first place! 
If a task manager is too cumber- 
some to use, you won’t bother with it. 
Make it as easy as possible to get a 
task out of your head and into the sys- 
tem. Make your task manager a 
keystroke or click away, and you’ll 
find yourself using it more often. 


Don’t Get Overwhelmed 

Keep your TODO list short so that 
you don’t get overwhelmed by all the 
things you need to do. Ruthlessly 
prune TODO items you no longer 
have to do or are no longer interested 
in doing. Delete or archive completed 
tasks so that they don’t clutter your 
main task list. 

TODO items can be intimidating. 
“Write a novel” is an example of a task 
that can be difficult to start. Make sure 
your TODO items are small enough to 
work on. I usually break my tasks 
down into subtasks I can do in one sit- 
ting. Breaking these tasks down also 
makes it easier to stop procrastinating, 
because there’s always something small 
to work on. 


9OMOCTOBER 2005 WWW.LINUXJOURNAL.COM 


Fill in the Cracks 

Make a system you can trust. Ensure that none of the tasks fall 
through the cracks. Make your reminder system the first thing 
that shows up after you log on or start your browser. Set aside 
time to review all of your tasks regularly. 

If your task manager is easy to use, you’ll trust it with more 
tasks. Writing down all of your tasks in a reliable system 
means you don’t have to worry about forgetting anything—as 
long as you don’t forget to check! 


Hack Your System 

The way you keep track of tasks probably will change as you 
come up with new ideas or read about other people’s experi- 
ences. Don’t be afraid to improve your system. Instead of mak- 
ing a giant step to a brand-new methodology, however, break 
changes down into incremental improvements. That way, you 
give yourself time to make it a habit. 

Don’t spend too much time tweaking your system, though! 
One way to manage this impulse is to find a community of 
like-minded people. That way, you can use their hacks and cus- 
tomizations without having to spend a lot of time coming up 
with your own. The trick is to find a personal information man- 
ager that fits the way you work and can be extended as you 
experiment with new ways of working. 


A Truly Personalized Personal Information Manager 
I went through the whole spectrum of 


Not only that, but it also intelligently picks up information 
from whatever I’m looking at, automatically creating a hyper- 
link back to the file, e-mail, Web page or even IRC session 
(Figure 2). Even newbies can add support for new tools, thanks 
to extensive examples. Planner.el’s ability to hyperlink to my 
mail messages is the only way I can impose order on the thou- 
sands of messages in my mail archive! 

I like reviewing my week to see what I have accomplished. 
Because it’s easy to view completed tasks, I can write accom- 


Taming the [ODO 


You want to keep things simple 


b-:-— private. tami ng-the- tod een 


Figure 1. You can create a task using a small text prompt in your regular editing 


window. 


personal information managers before I 
found something that works for me. 
I’m absolutely crazy about Planner.el, a 
personal information manager that’s 
extremely customizable. Id like to 
share some of the things I love about it 
with you so that you can see how per- 
sonal work style affects how you plan. 


I spend most of my time working 
with text files in the Emacs text editing 
environment. Because Emacs is so 
extensible, it has accumulated a lot of 
useful modules along the way, includ- 
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browsers, Internet relay chat (IRC) 
clients and even instant messengers. I 
can program, surf, chat and check mail 
within Emacs. Emacs itself runs on 
GNU/Linux, Microsoft Windows and 
Mac OS X and is surprisingly easy to 
learn. 

Planner.el is built into my main 
working environment, making it only a 
keystroke away. Because most of my 
tasks are based on what I’m looking at, 
I really appreciate how Planner.el stays 
out of my way. When I create a task, a 
small text prompt shows up at the bot- 
tom of my screen (Figure 1). I don’t 
get distracted by pop-ups or switching 
to another application. I simply type 
the task description in, tag it with a 
project or two and get back to work. 
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Taming the TODO 


Figure 2. Hyperlinked tasks give you an easy way to refer back to information 
related to a task, whether in the form of a file, mail message, Web page or 


IRC session. 


Figure 3. Publishing to the Net lets you check your tasks from any 


platform, anywhere. 


plishment reports without struggling to remember what I did 
the other day. Seeing a lot of crossed-out tasks for today also is 
a great morale booster. As a nifty bonus, I can keep detailed 
logs of how much time I spend on each task or project—great 
for billing time, improving my time estimates or simply finding 
out how (un)productive I am each day. 
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Manageable, not Overwhelming 

I like keeping my task list short. I typically have fewer than ten 
tasks on my task list for any given day. I like scheduling tasks 
for particular days and organizing them according to projects, 
keeping my daily task list small and manageable. When I feel 
particularly productive, it’s easy to reschedule more tasks onto 
today’s page. 

I break tasks down into bite-size bits to simplify keeping 
track of my progress and to motivate me to work. When tasks 
are of a manageable size, they’re much easier to work on. 
Instead of goofing off, I find myself picking the next small 
task from my list and working on it. 


Trustworthy 

I need a system that can keep track of small tasks as well as large 
projects. Because Planner.el is only a keystroke away and I use it 
for all of my tasks, I trust that it holds all the things I need to 
remember. I made Planner.el the first thing that shows up when I 
turn on my computer, and I check it at least once a day. Knowing 
that all of my reminders are safe and can be checked easily from 
one place definitely takes a load off my mind. 

It’s also easy for me to back up my files. Because 
Planner.el uses plain-text files, I don’t have to worry about 
corrupted data. If some experimental code makes Planner.el 
unusable for me, I still can use any text editor to manage my 
plans. In addition, it’s easy to publish my task list and notes as 
HTML (Figure 3), so if something happens to my laptop, I can 
check my TODOs using any computer with Net access. 


Extremely Customizable 

My method of planning has really changed over the years. I 
went from micromanaging my schedule by assigning specific 
times to tasks to keeping an unsorted list on my day page. I 
tried both keeping one big list of tasks and using projects to 
group together related tasks. Sometimes I think up weird 
things, too, such as having my computer automatically display 
a fortune cookie whenever I finish a task. 

This is where Planner.el really shines. Because it’s built on 
top of Emacs, I can change anything I want through a simple, 
easy-to-learn programming language. I’ve tweaked it to fit not 
only my planning style but also my little quirks. Although my 
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Although a request 
tracker requires a lot of 
setup time and effort, 
you reap the benefits 
of a solid project 
Management system. 


planning style has changed much in the 
past three years, being able to replace 
bits of Planner.el and add new features 
has made it possible for Planner.el to 
grow along with me. 


Things to Remember 

There are many ways to manage your 
tasks, so spend some time finding 
one that fits you. Here are a few 
things to remember: 


@ Make it as easy as possible. Use 
keyboard shortcuts and scripts to 
simplify task creation and review. 


m@ Don’t get overwhelmed. Keep your 
task list short and simple. Don’t 
drown in hundreds of TODOs or 
choke on intimidating tasks. 


@ Fill in the cracks. Put all of your 
important tasks in there. If you can, 
put minor tasks in as well. Check 
your list regularly. 


m™ Hack your system. Keep an eye out 
for ways to improve your way of 
planning. Don’t spend too much 
time hacking your system and not 
enough time actually accomplishing 
your TODOs, however. 


Have fun! 
Resources for this article: 
www.linuxjournal.com/article/8461.4 


Sandra Jean Chua—or Sacha, 

as she commonly is 

known—maintains Planner.el 

and is absolutely crazy 

me bout it. Her blog and TODO 

list is at sacha.free.net.ph. Write her at 
sacha@free.net.ph with your productivity 
tips and way of working! 


LETTERS CONTINUED FROM PAGE 8 


sed Tip 

——— 
I read with great interest the article by 
Larry Richardson entitled “Text 
Manipulation with sed” [July 2005]. We’re 
in the early stages of porting a MS Access 
application to Linux using MySQL and 

Qt to build the GUI. I’ve recently been 
working with sed, so the article hit the 
spot, so to speak. 


One thing I noted missing from the article 
is the fact that when writing commands 
that substitute one string with another, 

for example, using slashes to separate 

the from string and the to string is only 
one option. Actually, any unique character 
could be used as a separator. In the 
example on page 81 (first column, 

second example), which reads: 


cat customer.txt \ 
sed -e 's/Sam Jones/Samuel Jones/' > 
customer.txt 


The slashes could be replaced by the plus 
(+) sign (or most any other unique character) 
as in: 


cat customer.txt \ 
sed -e 's+Sam Jones+Samuel Jones+' > 
customer.txt 


I’ve found this useful when I’m editing a file 
containing paths, such as /etc/exports. In this 
case, the command: 

cat /etc/exports | 

sed -e 's/\/windows/\/winNT/' 
>/etc/exports 


reduces to: 


cat /etc/exports | 


sed -e 's+/windows+/winNT+' 
>/etc/exports 
Bill Lugg 


Patent Searching 
Se 
I’ve liked open software since the time it 
had no name yet, when you were nearly a 
hacker because you had a 1,200-baud 
modem and a dial-up connection to some 
friend’s BBS. 


I’m writing from the old continent, where a 
battle is being played between software com- 
panies who want the ratification by the EU 
of the right for software patents and develop- 
ers who don’t want to have their mind and 
ideas limited. 


And now the question: I suppose that any 
person should be granted the same rights 
as anyone else, but in the fight between 
open software and non-open software there 
is a disparity. Any company developing 
non-open software can check any open 
software to look for similar, copyright 
infringing, code. The same right is not 
granted to the counterpart. How can we 
be sure that Microsoft, Oracle, SAP or 
anyone else, is not importing code and 
concepts from open-source software? 
Maybe this issue has been discussed many 
times, but I couldn’t find a definitive 
answer to it. 


Andrea Rui 


GPL violators do sometimes get caught. See 
gpl-violations.org for examples.—Ed. 


LDAP Question 

— | 
I just found your OpenLDAP article on 
www.linuxjournal.com [July 2005]. I 
notice you include the nis.schema, but you 
don’t use NIS at all, correct? Thanks for 
any Clarification. 


Jiann-Ming Su 


Craig Swanson replies: thank you for 
your interest in “OpenLDAP Everywhere 
Revisited”. You are correct that we are 
not using Network Information System 
(NIS). The nis.schema provides several 
essential attribute types that are used in 
our LDAP directory. For example, 
nis.schema defines the LDAP entries 

that take the place of fields in /etc/passwd. 
For background information on the history 
of NIS and LDAP, see the IETF RFC2307, 
“An Approach for Using LDAP as a 
Network Information Service” .§ 


We welcome your letters. Please submit “Letters to the 
Editor” to ljeditor@ssc.com or SSC/Editorial, PO Box 55549, 
Seattle, WA 98155-0549 USA. 
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AL.COM 


SOFTWARE INC. 


(613) 231-6599 


www.roaringpenguin.com/partners 


Linux Support Technician 


Aberdeen LLC, a growing Los Angeles area server 
manufacturer is looking for a Linux expert. 


Duties include software and hardware pre-sale 
and post-sale technical support along with system 
assembly, configuration and testing. Some 
technical writing. Linux certification a plus. Full 
time position with excellent salary and benefits. 


Fax or email resume: 
(562) 695-5570 
hr@aberdeeninc.com 


> ABERDEEN 


Who says penguins 
can't fly? 


http://store.linuxjournal.com/ 
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How will you Spend your extra time and money 7 


Secure 

Remote Control 
& Support 

for Linux 


Award-winning NetOp Remote Control for Linux provides secure, 
cross-platform, remote control, access and support. NetOp lets 
you view and control a remote PC's current desktop session, trans- 
fer and synchronize files, launch applications or chat with the 
remote user - just as if you were seated at that computer. 


NetOp 


RemoteControl 


> Cross-Platform support for Linux, Solaris, Mac OS X, & all 
Windows platforms 

> Advanced security including encryption, multiple passwords, 
even centralized authentication & authorization with the 
optional NetOp Security Server module 


NetOp and the red kite are registered trademarks of Danware Data A/S. Other brand and product names 
are S. 


# trademarks of their respective halders, 2003 Copyright Danware Data A/S. Allrights reserved 


Try it Free - www.CrossTecCorp.com Ke C70ssléc 


www.linuxjournal.com 
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The Universal Internet 
Time Source 


If your computer’s clock is wrong, scheduled tasks, such as backup and virus scans, might run during the day, 
or your log files could become worthless. The pool.ntp.org project has an answer. BY ADRIAN VON BIDDER 


n theory, setting a computer’s clock over the network is 
easy: simply send a query to the time server and get the 
current time in return. For low-precision usage on a trusted 
network, this process does indeed work fine, as demon- 
strated by the old UNIX “time” protocol. For today’s Internet, 
however, and for millisecond (ms) or even sub-ms precision, 
problems such as authentication, reliability of the time servers 
and network delays need to be considered. This is where the 
Network Time Protocol (NTP), with its reference implementa- 
tion, steps in. The specification and the reference implementa- 
tion are being written by Professor David Mills of the University 
of Delaware, his graduate students and many other volunteers. 

To allow everybody to use NTP to synchronise computers’ 
clocks over the public Internet, Prof. Mills has long maintained a 
list of public time servers. Most of these servers are operated by 
universities or national standardisation organisations. Today, this 
list is maintained by the NTP Public Services Project, under the 
umbrella of the Internet Systems Consortium. However, the 
growth of the Internet and the prevalence of small, cheap appli- 
ances, such as cable or DSL routers, with built-in NTP clients, 
lead to a rapidly growing load on these public time servers. One 
of the most famous cases involved a severe firmware problem in 
a range of such devices, resulting in more than 150Mbps of NTP 
traffic to the University of Wisconsin’s NTP server. 

After reading the discussion of one time server operator’s 
request to be taken off the public time servers list, I wondered 
if there was a better approach to this whole problem—instead 
of having tens of thousands of clients targeting one single time 
server, the load should be distributed on many different time 
servers all over the network. So I went ahead and created the 
original time.fortytwo.ch DNS round-robin in January 2003. 
The project quickly acquired many interested volunteers and 
was well received by Prof. Mills and his team. It soon became 
the pool.ntp.org project with a somewhat more official status. 


The Road Ahead 

During the next two years, the project continued to grow, thanks 
to all the people who mentioned it in various Web forums, 
HOWTO documents and the like. Today, the project consists of 
more than 300 servers, offering service to tens of thousands of 
clients, in a very rough estimate. Also, pool.ntp.org is now the 
default time server in several operating system distributions, 
including Debian GNU/Linux, NetBSD and Gentoo Linux. 

So far, the growth in servers could more or less match the 
growth of the user base of the project. However, the future 
remains challenging, and discussions on the project’s discus- 
sion mailing list have shown that the project needs to deal with 
an inherent conflict between providing easy service for as 
many clients as possible and assuring good quality of the time 


servers participating in the project. That aside, the big chal- 
lenges for the near and medium future are: 


m@ IPv6 integration. 


@ More automation—currently, I process server additions and 
removals mostly manually. 


m@ Better, more novice-friendly documentation on the Web. 
m@ Of course, we always need more servers too. 


@ And above all, we need to deal with abusive clients. In one 
example, the six worst clients were responsible for 25% of 
the traffic on one time server. 


Although the first three items are not technically difficult 
and the “getting more servers” plan should see a big leap ahead 
with the publication of this article, we don’t currently have a 
good plan to educate the hundreds of users with sub-optimally 
configured clients. Due to their number, they are a serious 
problem for the project. At the same time, the bandwidth per 
client is small enough that the big ISPs’ abuse departments are 
not prepared to help in any way. 

In the medium to long term, we will need to face the issue that 
DNS round-robin, as currently implemented, is not a good solution 
for load balancing on the scale of several hundred servers with a 
hundred thousand or more clients. Wide deployment of IP multi- 
cast together with the existing multicast support in ntpd would be a 
good solution to this problem, but obviously not one the NTP and 
pool.ntp.org crew can deploy on their own. Another possible solu- 
tion is to make the ntpd demon aware of the pool.ntp.org project 
and, in some generic way, similar such databases, and have the 
deemon configure itself to use such a resource. 

Finally, on a personal note, I honestly can say that it was 
fun to get this project started and see it grow, but I now see the 
need for somebody new, with fresh ideas, to take over from 
here. Indeed, as I write this, I am talking with several people 
about the project’s future, and I am certain that the involve- 
ment of a new “father figure” will do the project much good as 
new ideas are looked at and implemented by a new crew. 

Resources for this article: www.linuxjournal.com/article/ 
8454.4 


Adrian von Bidder graduated with a degree in computer sci- 
ence from the Federal Institute of Technology in Zurich, 
Switzerland, in 2004. He is running the pool.ntp.org project in 
his spare time. His day job is developing the SEPP e-mail 
encryption gateway at Onaras AG in Wettingen, Switzerland. 
He can be contacted at avbidder@fortytwo.ch. 
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Rackspace — Managed Hosting backed by Fanatical Support* 


Servers, data centers and bandwidth are not the key to hosting enterprise class Web sites and Web applications. 
At Rackspace, we believe hosting is a service, not just technology. 


Fanatical Support is our philosophy, our credo. It reflects our desire to bring responsiveness and value 
to everything we do for our customers. You will experience Fanatical Support from the moment we answer the 
phone and you begin to interact with our employees. 


Fanatical Support has made Rackspace the fastest-growing hosting company in the world. Call today to 
experience the difference with Fanatical Support at Rackspace. 
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2004 Linux Journal MANAGED BHOSTING 


Readers' Choice Award for 


“Favorite Web-Hosting Service” 1.888.571.8976 or visit us at www.rackspace.com 
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architecture makes it possible to create 
24 to 72 port modular fabrics which 
have lower latency than monolithic switches. They 
aggregate data modulo 24 instead of 12, improving nearest neighbor 

latency in fine grain problems and doubling the size of the largest three hop fat tree 
that can be built, from 288 to 576 ports. Larger fabrics can be created linking 576 port domains together. 
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A72 Port FasTree™ Configuration 


Working with PathScale's InfiniPath HTX Adapters, the number of hops required to move MPI messages 
between nodes is reduced, improving latency. The modular design makes them useful for SDR, DDR and 
future QDR InfiniBand fabrics, greatly extending their useful life. Please send email to fastree@microway.com 
to request our white paper entitled Low Latency Modular Switches for InfiniBand. 


Microway's QuadPuter® includes four AMD single or dual core Opteron™ processors, 1350 Watt redundant 
power supply, and up to 5 redundant, hot swap hard drives-all in 4U. One of the most powerful processing 
platforms in the HPC industry, QuadPuter can serve as a cluster node or a standalone supercomputer. 
Constructed with stainless steel, its RuggedRack™ architecture is designed to keep the processors and 
memory running cool and efficiently. The power supply exhaust does not mix with air in the motherboard 
chamber. Hard drives are cooled with external air and are front-mounted along with the power supply for 
easy access and removal. The RuggedRack” is available with an 8-way motherboard, dual-core Opterons 
and up to 128 GB of memory for power- and memory-hungry SMP applications. 


Visit us at SC2005 Seattle 


<4 QuadPuter® Navion™ with Hot Swap, Redundant Power & Hard Drives 
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